Selective mixing for entropy coding in video compression

ABSTRACT

An apparatus for decoding transform coefficients using an alphabet of transform coefficient tokens includes a memory and a processor. The processor is configured to execute instructions stored in the memory to select a first probability distribution corresponding to a first context, select a second probability distribution corresponding to a second context, and, in response to determining that the second probability distribution includes a probability for a transform coefficient token, mix the first probability distribution and the second probability distribution to generate a mixed probability and entropy decode, from an encoded bitstream, the transform coefficient token using the mixed probability. The first probability distribution is defined for all tokens of the alphabet. The second probability distribution is defined over a non-trivial partition of the tokens.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims priority to and the benefit of U.S. ProvisionalPatent Application Ser. No. 62/551,341, filed Aug. 29, 2017, the entiredisclosure of which is hereby incorporated by reference.

BACKGROUND

Digital video streams may represent video using a sequence of frames orstill images. Digital video can be used for various applicationsincluding, for example, video conferencing, high definition videoentertainment, video advertisements, or sharing of user-generatedvideos. A digital video stream can contain a large amount of data andconsume a significant amount of computing or communication resources ofa computing device for processing, transmission, or storage of the videodata. Various approaches have been proposed to reduce the amount of datain video streams, including compression and other encoding techniques.

Encoding based on motion estimation and compensation may be performed bybreaking frames or images into blocks that are predicted based on one ormore prediction blocks of reference frames. Differences (i.e., residualerrors) between blocks and prediction blocks are compressed and encodedin a bitstream. A decoder uses the differences and the reference framesto reconstruct the frames or images.

SUMMARY

An aspect is an apparatus for decoding transform coefficients using analphabet of transform coefficient tokens including a memory and aprocessor. The processor is configured to execute instructions stored inthe memory to select a first probability distribution corresponding to afirst context, select a second probability distribution corresponding toa second context, and, in response to determining that the secondprobability distribution includes a probability for a transformcoefficient token, mix the first probability distribution and the secondprobability distribution to generate a mixed probability and entropydecode, from an encoded bitstream, the transform coefficient token usingthe mixed probability. The first probability distribution is defined forall tokens of the alphabet. The second probability distribution isdefined over a non-trivial partition of the tokens.

Another aspect is a method for coding transform coefficients using analphabet of tokens. The method includes selecting a first probabilitydistribution corresponding to a first context and that is defined forsome tokens of the alphabet, selecting a second probability distributioncorresponding to a second context and that is defined over a non-trivialpartition of the tokens, and, in response to determining that the firstprobability distribution includes a probability for a token and thesecond probability distribution includes a second probability for thetoken, mixing the first probability distribution and the secondprobability distribution to generate a mixed probability and coding thetoken using the mixed probability.

Another aspect is an apparatus for decoding transform coefficients usingan alphabet of tokens organized in a coefficient token tree including amemory and a processor. The processor is configured to executeinstructions stored in the memory to select a first probabilitydistribution corresponding to a first context and defined for internalnodes of the coefficient token tree, select a second probabilitydistribution corresponding to a second context and defined for some, butnot all, internal nodes of the coefficient token tree, and decode atoken by decoding a first decision related to a first internal node ofthe coefficient token tree using a mixed probability. The mixedprobability is generated by mixing the first probability distributionand the second probability distribution.

These and other aspects of the present disclosure are disclosed in thefollowing detailed description of the embodiments, the appended claims,and the accompanying figures.

BRIEF DESCRIPTION OF THE DRAWINGS

The description herein refers to the accompanying drawings wherein likereference numerals refer to like parts throughout the several views.

FIG. 1 is a schematic of a video encoding and decoding system.

FIG. 2 is a block diagram of an example of a computing device that canimplement a transmitting station or a receiving station.

FIG. 3 is a diagram of a video stream to be encoded and subsequentlydecoded.

FIG. 4 is a block diagram of an encoder according to implementations ofthis disclosure.

FIG. 5 is a block diagram of a decoder according to implementations ofthis disclosure.

FIG. 6 is a diagram illustrating quantized transform coefficientsaccording to implementations of this disclosure.

FIG. 7 is a diagram of a coefficient token tree that can be used toentropy code blocks into a video bitstream according to implementationsof this disclosure.

FIG. 8 is a diagram of an example of a tree for binarizing a quantizedtransform coefficient according to implementations of this disclosure.

FIG. 9 is a flowchart diagram of a process for encoding a sequence ofsymbols according to an implementation of this disclosure.

FIG. 10 is a flowchart diagram of a process for decoding a sequence ofsymbols according to an implementation of this disclosure.

FIG. 11 is a diagram of an example of a binary tree of conditionalprobabilities according to an implementation of this disclosure.

FIG. 12 is a flowchart diagram of a process for entropy coding accordingto an implementation of this disclosure.

FIG. 13 is a flowchart diagram of a process for coding transformcoefficients using an alphabet of transform coefficient tokens accordingto an implementation of this disclosure.

FIG. 14 is a diagram of neighborhood templates for deriving a contextaccording to implementations of this disclosure.

DETAILED DESCRIPTION

As mentioned above, compression schemes related to coding video streamsmay include breaking images into blocks and generating a digital videooutput bitstream using one or more techniques to limit the informationincluded in the output. A received encoded bitstream can be decoded tore-create the blocks and the source images from the limited information.Encoding a video stream, or a portion thereof, such as a frame or ablock, can include using temporal or spatial similarities in the videostream to improve coding efficiency. For example, a current block of avideo stream may be encoded based on identifying a difference (residual)between the previously coded pixel values and those in the currentblock. In this way, only the residual and parameters used to generatethe residual need be added to the encoded bitstream. The residual may beencoded using a lossy quantization step.

As further described below, the residual block can be in the pixeldomain. The residual block can be transformed into the frequency domainresulting in a transform block of transform coefficients. The transformcoefficients can be quantized resulting into a quantized transform blockof quantized transform coefficients. The quantized coefficients can beentropy encoded and added to an encoded bitstream. A decoder can receivethe encoded bitstream, entropy decode the quantized transformcoefficients to reconstruct the original video frame.

Entropy coding is a technique for “lossless” coding that relies uponprobability models that model the distribution of values occurring in anencoded video bitstream. By using probability models based on a measuredor estimated distribution of values, entropy coding can reduce thenumber of bits required to represent video data close to a theoreticalminimum. In practice, the actual reduction in the number of bitsrequired to represent video data can be a function of the accuracy ofthe probability model, the number of bits over which the coding isperformed, and the computational accuracy of fixed-point arithmetic usedto perform the coding.

In an encoded video bitstream, many of the bits are used for one of twothings: either content prediction (e.g., inter mode/motion vectorcoding, intra prediction mode coding, etc.) or residual coding (e.g.,transform coefficients). Encoders may use techniques to decrease theamount of bits spent on coefficient coding. For example, a coefficienttoken tree (which may also be referred to as a binary token tree)specifies the scope of the value, with forward-adaptive probabilitiesfor each branch in this token tree. The token base value is subtractedfrom the value to be coded to form a residual then the block is codedwith fixed probabilities. A similar scheme with minor variationsincluding backward-adaptivity is also possible. Adaptive techniques canalter the probability models as the video stream is being encoded toadapt to changing characteristics of the data. In any event, a decoderis informed of (or has available) the probability model used to encodean entropy-coded video bitstream in order to decode the video bitstream.

As described above, entropy coding a sequence of symbols is typicallyachieved by using a probability model to determine a probability p forthe sequence and then using binary arithmetic coding to map the sequenceto a binary codeword at the encoder and to decode that sequence from thebinary codeword at the decoder. The length (i.e., number of bits) of thecodeword is given by −log(p). The efficiency of entropy coding can bedirectly related to the probability model. Throughout this document, logdenotes the logarithm function to base 2 unless specified otherwise.

A model, as used herein, can be, or can be a parameter in, a lossless(entropy) coding. A model can be any parameter or method that affectsprobability estimation for entropy coding. For example, a model candefine the probability to be used to encode and decode the decision atan internal node in a token tree (such as described with respect to FIG.7 below). In such a case, the two-pass process to learn theprobabilities for a current frame may be simplified to a single-passprocess by mixing multiple models as described herein. In anotherexample, a model may define a certain context derivation method. In sucha case, implementations according to this disclosure can be used toautomatically mix coding probabilities generated by a multitude of suchmethods. In yet another example, a model may define a completely newlossless coding algorithm.

A purpose of context modeling is to obtain probability distributions fora subsequent entropy coding engine, such as arithmetic coding, Huffmancoding, and other variable-length-to-variable-length coding engines. Toachieve good compression performance, a large number of contexts may berequired. For example, some video coding systems can include hundreds oreven thousands of contexts for transform coefficient coding alone. Eachcontext can correspond to a probability distribution.

A probability distribution can be learnt by a decoder and/or included inthe header of a frame to be decoded.

Learnt can mean that an entropy coding engine of a decoder can adapt theprobability distributions (i.e., probability models) of a context modelbased on decoded frames. For example, the decoder can have available aninitial probability distribution that the decoder (e.g., the entropycoding engine of the decoder) can continuously update as the decoderdecodes additional frames. The updating of the probability models caninsure that the initial probability distribution is updated to reflectthe actual distributions in the decoded frames.

Including a probability distribution in the header can instruct thedecoder to use the included probability distribution for decoding thenext frame, given the corresponding context. A cost (in bits) isassociated with including each probability distribution in the header.For example, in a coding system that includes 3000 contexts and thatencodes a probability distribution (coded as an integer value between 1and 255) using 8 bits, 24,000 bits are added to the encoded bitstream.These bits are overhead bits. Some techniques can be used to reduce thenumber of overhead bits. For example, the probability distributions forsome, but not all, of the contexts can be included. For example,prediction schemes can also be used to reduce the overhead bits. Evenwith these overhead reduction techniques, the overhead is non-zero.

A key design challenge or problem in context modeling is to balancebetween two conflicting objectives, which are further described below,namely: 1) improving compression performance by adding more contexts and2) reducing the overhead cost associated with contexts. The problem isparticularly relevant in cases where multi-symbol, non-binary alphabetsare involved due, in part, to the fact that the overhead associated witha context grows as the alphabet size grows.

Implementations according to this disclosure use selective mixing sothat contexts can be added while limiting the overhead associated withthe added contexts. A context can define a probability distributionover, for example, the alphabet of tokens used to encode transformcoefficients. In selective mixing, a first context model can be used todetermine a first probability distribution defined for all tokens and asecond context can be used to determine a second probabilitydistribution for more frequent tokens. The number of the more frequenttokens is less than the number of all tokens. In coding a coefficient,the selective mixing mixes the first and the second probabilitydistributions for the more frequent tokens and uses the firstprobability distribution for the remaining tokens.

Implementations according to this disclosure can use selective mixing ofprobability models. Mixing models can be used for encoding any valuethat is encoded using entropy coding. For example, two or moreprobability models can be mixed in order to entropy code quantizedtransform coefficients. The benefits of implementations according tothis disclosure include reduced overhead associated with contexts andimproved compression performance. Additionally, using selective mixing,context modeling in a coding system can be designed such that somecontexts can be derived from contextual information that affects theoverall distribution over an alphabet E while other contexts can bederived from contextual information that affects only a portion of thedistribution over the alphabet E thereby reducing the overheadassociated with contexts as compared to coding systems that do not useselective mixing.

Mixing for entropy coding in video compression is described herein firstwith reference to a system in which the teachings may be incorporated.

FIG. 1 is a schematic of a video encoding and decoding system 100. Atransmitting station 102 can be, for example, a computer having aninternal configuration of hardware such as that described in FIG. 2.However, other suitable implementations of the transmitting station 102are possible. For example, the processing of the transmitting station102 can be distributed among multiple devices.

A network 104 can connect the transmitting station 102 and a receivingstation 106 for encoding and decoding of the video stream. Specifically,the video stream can be encoded in the transmitting station 102 and theencoded video stream can be decoded in the receiving station 106. Thenetwork 104 can be, for example, the Internet. The network 104 can alsobe a local area network (LAN), wide area network (WAN), virtual privatenetwork (VPN), cellular telephone network, or any other means oftransferring the video stream from the transmitting station 102 to, inthis example, the receiving station 106.

The receiving station 106, in one example, can be a computer having aninternal configuration of hardware such as that described in FIG. 2.However, other suitable implementations of the receiving station 106 arepossible. For example, the processing of the receiving station 106 canbe distributed among multiple devices.

Other implementations of the video encoding and decoding system 100 arepossible. For example, an implementation can omit the network 104. Inanother implementation, a video stream can be encoded and then storedfor transmission, at a later time, to the receiving station 106 or anyother device having memory. In one implementation, the receiving station106 receives (e.g., via the network 104, a computer bus, and/or somecommunication pathway) the encoded video stream and stores the videostream for later decoding. In an example implementation, a real-timetransport protocol (RTP) is used for transmission of the encoded videoover the network 104. In another implementation, a transport protocolother than RTP may be used, e.g., an HTTP-based video streamingprotocol.

When used in a video conferencing system, for example, the transmittingstation 102 and/or the receiving station 106 may include the ability toboth encode and decode a video stream as described below. For example,the receiving station 106 could be a video conference participant whoreceives an encoded video bitstream from a video conference server(e.g., the transmitting station 102) to decode and view and furtherencodes and transmits its own video bitstream to the video conferenceserver for decoding and viewing by other participants.

FIG. 2 is a block diagram of an example of a computing device 200 thatcan implement a transmitting station or a receiving station. Forexample, the computing device 200 can implement one or both of thetransmitting station 102 and the receiving station 106 of FIG. 1. Thecomputing device 200 can be in the form of a computing system includingmultiple computing devices, or in the form of a single computing device,for example, a mobile phone, a tablet computer, a laptop computer, anotebook computer, a desktop computer, and the like.

A CPU 202 in the computing device 200 can be a central processing unit.Alternatively, the CPU 202 can be any other type of device, or multipledevices, capable of manipulating or processing information now existingor hereafter developed. Although the disclosed implementations can bepracticed with a single processor as shown, e.g., the CPU 202,advantages in speed and efficiency can be achieved using more than oneprocessor.

A memory 204 in the computing device 200 can be a read-only memory (ROM)device or a random access memory (RAM) device in an implementation. Anyother suitable type of storage device can be used as the memory 204. Thememory 204 can include code and data 206 that is accessed by the CPU 202using a bus 212. The memory 204 can further include an operating system208 and application programs 210, the application programs 210 includingat least one program that permits the CPU 202 to perform the methodsdescribed here. For example, the application programs 210 can includeapplications 1 through N, which further include a video codingapplication that performs the methods described here. The computingdevice 200 can also include a secondary storage 214, which can, forexample, be a memory card used with a computing device 200 that ismobile. Because the video communication sessions may contain asignificant amount of information, they can be stored in whole or inpart in the secondary storage 214 and loaded into the memory 204 asneeded for processing.

The computing device 200 can also include one or more output devices,such as a display 218. The display 218 may be, in one example, a touchsensitive display that combines a display with a touch sensitive elementthat is operable to sense touch inputs. The display 218 can be coupledto the CPU 202 via the bus 212. Other output devices that permit a userto program or otherwise use the computing device 200 can be provided inaddition to or as an alternative to the display 218. When the outputdevice is or includes a display, the display can be implemented invarious ways, including by a liquid crystal display (LCD), a cathode-raytube (CRT) display or light emitting diode (LED) display, such as anorganic LED (OLED) display.

The computing device 200 can also include or be in communication with animage-sensing device 220, for example, a camera or any otherimage-sensing device 220 now existing or hereafter developed that cansense an image such as the image of a user operating the computingdevice 200. The image-sensing device 220 can be positioned such that itis directed toward the user operating the computing device 200. In anexample, the position and optical axis of the image-sensing device 220can be configured such that the field of vision includes an area that isdirectly adjacent to the display 218 and from which the display 218 isvisible.

The computing device 200 can also include or be in communication with asound-sensing device 222, for example, a microphone or any othersound-sensing device now existing or hereafter developed that can sensesounds near the computing device 200. The sound-sensing device 222 canbe positioned such that it is directed toward the user operating thecomputing device 200 and can be configured to receive sounds, forexample, speech or other utterances, made by the user while the useroperates the computing device 200.

Although FIG. 2 depicts the CPU 202 and the memory 204 of the computingdevice 200 as being integrated into a single unit, other configurationscan be utilized. The operations of the CPU 202 can be distributed acrossmultiple machines (each machine having one or more of processors) thatcan be coupled directly or across a local area or other network. Thememory 204 can be distributed across multiple machines such as anetwork-based memory or memory in multiple machines performing theoperations of the computing device 200. Although depicted here as asingle bus, the bus 212 of the computing device 200 can be composed ofmultiple buses. Further, the secondary storage 214 can be directlycoupled to the other components of the computing device 200 or can beaccessed via a network and can comprise a single integrated unit such asa memory card or multiple units such as multiple memory cards. Thecomputing device 200 can thus be implemented in a wide variety ofconfigurations.

FIG. 3 is a diagram of an example of a video stream 300 to be encodedand subsequently decoded. The video stream 300 includes a video sequence302. At the next level, the video sequence 302 includes a number ofadjacent frames 304. While three frames are depicted as the adjacentframes 304, the video sequence 302 can include any number of adjacentframes 304. The adjacent frames 304 can then be further subdivided intoindividual frames, e.g., a frame 306. At the next level, the frame 306can be divided into a series of segments 308 or planes. The segments 308can be subsets of frames that permit parallel processing, for example.The segments 308 can also be subsets of frames that can separate thevideo data into separate colors. For example, the frame 306 of colorvideo data can include a luminance plane and two chrominance planes. Thesegments 308 may be sampled at different resolutions.

Whether or not the frame 306 is divided into the segments 308, the frame306 may be further subdivided into blocks 310, which can contain datacorresponding to, for example, 16×16 pixels in the frame 306. The blocks310 can also be arranged to include data from one or more segments 308of pixel data. The blocks 310 can also be of any other suitable sizesuch as 4×4 pixels, 8×8 pixels, 16×8 pixels, 8×16 pixels, 16×16 pixelsor larger.

FIG. 4 is a block diagram of an encoder 400 in accordance withimplementations of this disclosure. The encoder 400 can be implemented,as described above, in the transmitting station 102 such as by providinga computer software program stored in memory, for example, the memory204. The computer software program can include machine instructionsthat, when executed by a processor such as the CPU 202, cause thetransmitting station 102 to encode video data in the manner describedherein. The encoder 400 can also be implemented as specialized hardwareincluded in, for example, the transmitting station 102. The encoder 400has the following stages to perform the various functions in a forwardpath (shown by the solid connection lines) to produce an encoded orcompressed bitstream 420 using the video stream 300 as input: anintra/inter prediction stage 402, a transform stage 404, a quantizationstage 406, and an entropy encoding stage 408. The encoder 400 may alsoinclude a reconstruction path (shown by the dotted connection lines) toreconstruct a frame for encoding of future blocks. In FIG. 4, theencoder 400 has the following stages to perform the various functions inthe reconstruction path: a dequantization stage 410, an inversetransform stage 412, a reconstruction stage 414, and a loop filteringstage 416. Other structural variations of the encoder 400 can be used toencode the video stream 300.

When the video stream 300 is presented for encoding, the frame 306 canbe processed in units of blocks. At the intra/inter prediction stage402, a block can be encoded using intra-frame prediction (also calledintra-prediction) or inter-frame prediction (also calledinter-prediction), or a combination of both. In any case, a predictionblock can be formed. In the case of intra-prediction, all or a part of aprediction block may be formed from samples in the current frame thathave been previously encoded and reconstructed. In the case ofinter-prediction, all or part of a prediction block may be formed fromsamples in one or more previously constructed reference framesdetermined using motion vectors.

Next, still referring to FIG. 4, the prediction block can be subtractedfrom the current block at the intra/inter prediction stage 402 toproduce a residual block (also called a residual). The transform stage404 transforms the residual into transform coefficients in, for example,the frequency domain using block-based transforms. Such block-basedtransforms include, for example, the Discrete Cosine Transform (DCT) andthe Asymmetric Discrete Sine Transform (ADST). Other block-basedtransforms are possible. Further, combinations of different transformsmay be applied to a single residual. In one example of application of atransform, the DCT transforms the residual block into the frequencydomain where the transform coefficient values are based on spatialfrequency. The lowest frequency (DC) coefficient at the top-left of thematrix and the highest frequency coefficient at the bottom-right of thematrix. It is worth noting that the size of a prediction block, andhence the resulting residual block, may be different from the size ofthe transform block. For example, the prediction block may be split intosmaller blocks to which separate transforms are applied.

The quantization stage 406 converts the transform coefficients intodiscrete quantum values, which are referred to as quantized transformcoefficients, using a quantizer value or a quantization level. Forexample, the transform coefficients may be divided by the quantizervalue and truncated. The quantized transform coefficients are thenentropy encoded by the entropy encoding stage 408. Entropy coding may beperformed using any number of techniques, including token and binarytrees. The entropy-encoded coefficients, together with other informationused to decode the block, which may include for example the type ofprediction used, transform type, motion vectors and quantizer value, arethen output to the compressed bitstream 420. The information to decodethe block may be entropy coded into block, frame, slice and/or sectionheaders within the compressed bitstream 420. The compressed bitstream420 can also be referred to as an encoded video stream or encoded videobitstream, and the terms will be used interchangeably herein.

The reconstruction path in FIG. 4 (shown by the dotted connection lines)can be used to ensure that both the encoder 400 and a decoder 500(described below) use the same reference frames and blocks to decode thecompressed bitstream 420. The reconstruction path performs functionsthat are similar to functions that take place during the decodingprocess that are discussed in more detail below, including dequantizingthe quantized transform coefficients at the dequantization stage 410 andinverse transforming the dequantized transform coefficients at theinverse transform stage 412 to produce a derivative residual block (alsocalled a derivative residual). At the reconstruction stage 414, theprediction block that was predicted at the intra/inter prediction stage402 can be added to the derivative residual to create a reconstructedblock. The loop filtering stage 416 can be applied to the reconstructedblock to reduce distortion such as blocking artifacts.

Other variations of the encoder 400 can be used to encode the compressedbitstream 420. For example, a non-transform based encoder 400 canquantize the residual signal directly without the transform stage 404for certain blocks or frames. In another implementation, an encoder 400can have the quantization stage 406 and the dequantization stage 410combined into a single stage.

FIG. 5 is a block diagram of a decoder 500 in accordance withimplementations of this disclosure. The decoder 500 can be implementedin the receiving station 106, for example, by providing a computersoftware program stored in the memory 204. The computer software programcan include machine instructions that, when executed by a processor suchas the CPU 202, cause the receiving station 106 to decode video data inthe manner described in FIGS. 8 and 9 below. The decoder 500 can also beimplemented in hardware included in, for example, the transmittingstation 102 or the receiving station 106. The decoder 500, similar tothe reconstruction path of the encoder 400 discussed above, includes inone example the following stages to perform various functions to producean output video stream 516 from the compressed bitstream 420: an entropydecoding stage 502, a dequantization stage 504, an inverse transformstage 506, an intra/inter-prediction stage 508, a reconstruction stage510, a loop filtering stage 512 and a deblocking filtering stage 514.Other structural variations of the decoder 500 can be used to decode thecompressed bitstream 420.

When the compressed bitstream 420 is presented for decoding, the dataelements within the compressed bitstream 420 can be decoded by theentropy decoding stage 502 to produce a set of quantized transformcoefficients. The dequantization stage 504 dequantizes the quantizedtransform coefficients (e.g., by multiplying the quantized transformcoefficients by the quantizer value), and the inverse transform stage506 inverse transforms the dequantized transform coefficients using theselected transform type to produce a derivative residual that can beidentical to that created by the inverse transform stage 412 in theencoder 400. Using header information decoded from the compressedbitstream 420, the decoder 500 can use the intra/inter-prediction stage508 to create the same prediction block as was created in the encoder400, e.g., at the intra/inter prediction stage 402. At thereconstruction stage 510, the prediction block can be added to thederivative residual to create a reconstructed block. The loop filteringstage 512 can be applied to the reconstructed block to reduce blockingartifacts. Other filtering can be applied to the reconstructed block. Inan example, the deblocking filtering stage 514 is applied to thereconstructed block to reduce blocking distortion, and the result isoutput as an output video stream 516. The output video stream 516 canalso be referred to as a decoded video stream, and the terms will beused interchangeably herein.

Other variations of the decoder 500 can be used to decode the compressedbitstream 420. For example, the decoder 500 can produce the output videostream 516 without the deblocking filtering stage 514. In someimplementations of the decoder 500, the deblocking filtering stage 514is applied before the loop filtering stage 512. Additionally, oralternatively, the encoder 400 includes a deblocking filtering stage inaddition to the loop filtering stage 416.

FIG. 6 is a diagram 600 illustrating quantized transform coefficientsaccording to implementations of this disclosure. The diagram 600 depictsa current block 620, a scan order 602, a quantized transform block 604,a non-zero map 606, an end-of-block map 622, and a sign map 626. Thecurrent block 620 is illustrated as a 4×4 block. However, any block sizeis possible. For example, the current block can have a size (i.e.,dimensions) of 4×4, 8×8, 16×16, 32×32, or any other square orrectangular block size. The current block 620 can be a block of acurrent frame. In another example, the current frame may be partitionedinto segments (such as the segments 308 of FIG. 3), tiles, or the like,each including a collection of blocks, where the current block is ablock of the partition.

The quantized transform block 604 can be a block of size similar to thesize of the current block 620. The quantized transform block 604includes non-zero coefficients (e.g., a coefficient 608) and zerocoefficients (e.g., a coefficient 610). As described above, thequantized transform block 604 contains quantized transform coefficientsfor the residual block corresponding to the current block 620. Also asdescribed above, the quantized transform coefficients are entropy codedby an entropy-coding phase, such as the entropy coding stage 408 of FIG.4.

Entropy coding a quantized transform coefficient can involve theselection of a context model (also referred to as probability contextmodel, probability model, model, and context) which provides estimatesof conditional probabilities for coding the binary symbols of abinarized transform coefficient as described below with respect to FIG.7. When entropy coding a quantized transform coefficient, additionalinformation may be used as the context for selecting a context model.For example, the magnitudes of the previously coded transformcoefficients can be used, at least partially, for determining aprobability model.

To encode a transform block, a video coding system may traverse thetransform block in a scan order and encode (e.g., entropy encode) thequantized transform coefficients as the quantized transform coefficientsare respectively traversed (i.e., visited). In a zig-zag scan order,such as the scan order 602, the top left corner of the transform block(also known as the DC coefficient) is first traversed and encoded, thenext coefficient in the scan order (i.e., the transform coefficientcorresponding to the location labeled “1”) is traversed and encoded, andso on. In the zig-zag scan order (i.e., scan order 602), some quantizedtransform coefficients above and to the left of a current quantizedtransform coefficient (e.g., a to-be-encoded transform coefficient) aretraversed first. Other scan orders are possible. A one-dimensionalstructure (e.g., an array) of quantized transform coefficients canresult from the traversal of the two-dimensional quantized transformblock using the scan order.

In some examples, encoding the quantized transform block 604 can includedetermining the non-zero map 606, which indicates which quantizedtransform coefficients of the quantized transform block 604 are zero andwhich are non-zero. A non-zero coefficient and a zero coefficient can beindicated with values one (1) and zero (0), respectively, in thenon-zero map. For example, the non-zero map 606 includes a non-zero 607at Cartesian location (0, 0) corresponding to the coefficient 608 and azero 608 at Cartesian location (2, 0) corresponding to the coefficient610.

In some examples, encoding the quantized transform block 604 can includegenerating and encoding the end-of-block map 622. The end-of-block mapindicates whether a non-zero quantized transform coefficient of thequantized transform block 604 is the last non-zero coefficient withrespect to a given scan order. If a non-zero coefficient is not the lastnon-zero coefficient in the transform block, then it can be indicatedwith the binary bit 0 (zero) in the end-of-block map. If, on the otherhand, a non-zero coefficient is the last non-zero coefficient in thetransform block, then it can be indicated with the binary value 1 (one)in the end-of-block map. For example, as the quantized transformcoefficient corresponding to the scan location 11 (i.e., the lastnon-zero quantized transform coefficient 628) is the last non-zerocoefficient of the quantized transform block 604, it is indicated withthe end-of-block value 624 of 1 (one); all other non-zero transformcoefficients are indicated with a zero.

In some examples, encoding the quantized transform block 604 can includegenerating and encoding the sign map 626. The sign map 626 indicateswhich non-zero quantized transform coefficients of the quantizedtransform block 604 have positive values and which quantized transformcoefficients have negative values. Transform coefficients that are zeroneed not be indicated in the sign map. The sign map 626 illustrates thesign map for the quantized transform block 604. In the sign map,negative quantized transform coefficients can be indicated with a −1 andpositive quantized transform coefficients can be indicated with a 1.

FIG. 7 is a diagram of a coefficient token tree 700 that can be used toentropy code blocks into a video bitstream according to implementationsof this disclosure. The coefficient token tree 700 is referred to as abinary tree because, at each node of the tree, one of two branches mustbe taken (i.e., traversed). The coefficient token tree 700 includes aroot node 701 and a node 703 corresponding, respectively, to the nodeslabeled A and B.

As described above with respect to FIG. 6, when an end-of-block (EOB)token is detected for a block, coding of coefficients in the currentblock can terminate and the remaining coefficients in the block can beinferred to be zero. As such, the coding of EOB positions can be anessential part of coefficient in a video coding system.

In some video coding systems, a binary decision determining whether (ornot) a current token is equal to the EOB token of the current block iscoded immediately after an nonzero coefficient is decoded or at thefirst scan position (DC). In an example, for a transform block of sizeM×N, where M denotes the number of columns and N denotes the number ofrows in the transform block, the maximum number of times of codingwhether a current token is equal to the EOB token is equal to M×N. M andN can take values, such as the values 2, 4, 8, 16, 32, and 64. Asdescribed below, the binary decision corresponds to the coding of a “1”bit corresponding to the decision to move from the root node 701 to thenode 703 in the coefficient token tree 700. Herein, “coding a bit” canmean the outputting or generating of a bit in the codeword representinga transform coefficient being encoded. Similarly, “decoding a bit” canmean the reading (such as from an encoded bitstream) of a bit of thecodeword corresponding to a quantized transform coefficient beingdecoded such that the bit corresponds to a branch being traversed in thecoefficient token tree.

Using the coefficient token tree 700, a string of binary digits isgenerated for a quantized coefficient (e.g., the coefficients 608, 610of FIG. 6) of the quantized transform block (such as the quantizedtransform block 604 of FIG. 6).

In an example, the quantized coefficients in an N×N block (e.g.,quantized transform block 604) are organized into a 1D (one-dimensional)array (herein, an array u) following a prescribed scan order (e.g., thescan order 602 of FIG. 6). N can be 4, 8, 16, 32, or any other value.The quantized coefficient at the i^(th) position of the 1D array can bereferred as u[i], where i=0, . . . , N*N−1. The starting position of thelast run of zeroes in u[i], . . . , u[N*N−1] can be denoted as eob. Inthe case where when u[N*N−1] is not zero, the eob can be set to thevalue N*N. That is, if the last coefficient of the 1D array u is notzero, then eob can be set to the value N*N. Using the examples of FIG.6, the 1D array u can have the entries u[ ]=[−6, 0, −1, 0, 2, 4, 1, 0,0, 1, 0, −1, 0, 0, 0, 0]. The values at each of the u[i]s is a quantizedtransform coefficient. The quantized transform coefficients of the 1Darray u may also be referred herein simply as “coefficients” or“transform coefficients.” The coefficient at position i=0 (i.e.,u[0]=−6) corresponds to the DC coefficient. In this example, the eob isequal to 12 because there are no non-zero coefficients after the zerocoefficient at position 12 of the 1D array u.

To encode and decode the coefficients u[i], . . . , u[N*N−1], for i=0 toN*N−1, a token t[i] is generated at each position i<=eob. The tokent[i], for i<eob, can be indicative of the size and/or size range of thecorresponding quantized transform coefficient at u[i]. The token for thequantized transform coefficient at eob can be an EOB_TOKEN, which is atoken that indicates that the 1D array u contains no non-zerocoefficients following the eob position (inclusive). That is,t[eob]=EOB_TOKEN indicates the EOB position of the current block. TableI provides a listing of an example of token values, excluding theEOB_TOKEN, and their corresponding names according to an implementationof this disclosure.

TABLE I Token Name of Token 0 ZERO_TOKEN 1 ONE_TOKEN 2 TWO_TOKEN 3THREE_TOKEN 4 FOUR_TOKEN 5 DCT_VAL_CAT1 (5, 6) 6 DCT_VAL_CAT2 (7-10) 7DCT_VAL_CAT3 (11-18) 8 DCT_VAL_CAT4 (19-34) 9 DCT_VAL_CAT5 (35-66) 10DCT_VAL_CAT6 (67-2048)

In an example, quantized coefficient values are taken to be signed12-bit integers. To represent a quantized coefficient value, the rangeof 12-bit signed values can be divided into 11 tokens (the tokens 0-10in Table I) plus the end of block token (EOB_TOKEN). To generate a tokento represent a quantized coefficient value, the coefficient token tree700 can be traversed. The result (i.e., the bit string) of traversingthe tree can then be encoded into a bitstream (such as the bitstream 420of FIG. 4) by an encoder as described with respect to the entropyencoding stage 408 of FIG. 4.

The coefficient token tree 700 includes the tokens EOB_TOKEN (token702), ZERO_TOKEN (token 704), ONE_TOKEN (token 706), TWO_TOKEN (token708), THREE_TOKEN (token 710), FOUR_TOKEN (token 712), CAT1 (token 714that is DCT_VAL_CAT1 in Table I), CAT2 (token 716 that is DCT_VAL_CAT2in Table I), CAT3 (token 718 that is DCT_VAL_CAT3 in Table I), CAT4(token 720 that is DCT_VAL_CAT4 in Table I), CAT5 (token 722 that isDCT_VAL_CAT5 in Table I) and CAT6 (token 724 that is DCT_VAL_CAT6 inTable I). As can be seen, the coefficient token tree maps a singlequantized coefficient value into a single token, such as one of thetokens 704, 706, 708, 710 and 712. Other tokens, such as the tokens 714,716, 718, 720, 722 and 724, represent ranges of quantized coefficientvalues. For example, a quantized transform coefficient with a value of37 can be represented by the token DCT_VAL_CAT5 the token 722 in FIG. 7.

The base value for a token is defined as the smallest number in itsrange. For example, the base value for the token 720 is 19. Entropycoding identifies a token for each quantized coefficient and, if thetoken represents a range, can form a residual by subtracting the basevalue from the quantized coefficient. For example, a quantized transformcoefficient with a value of 20 can be represented by including the token720 and a residual value of 1 (i.e., 20 minus 19) in the encoded videobitstream to permit a decoder to reconstruct the original quantizedtransform coefficient. The end of block token (i.e., the token 702)signals that no further non-zero quantized coefficients remain in thetransformed block data.

In another example of token values for coefficient coding, Table 1 issplit up into two, where the first (head) set includes ZERO_TOKEN,ONE_NOEOB, ONE_EOB, TWO_NOEOB, and TWO_EOB; and the second (tail) setincludes TWO_TOKEN, THREE_TOKEN, FOUR_TOKEN, DCT_VAL_CAT1, DCT_VAL_CAT2,DCT_VAL_CAT3, DCT_VAL_CAT4, DCT_VAL_CAT5 and DCT_VAL_CAT6. The second(tail) set is used only if a TWO_EOB or TWO_NOEOB in the first (head)set is encoded or decoded. The tokens ONE_NOEOB and TWO_NOEOBcorrespond, respectively to the ONE_TOKEN and the TWO_TOKEN whentraversal of the coefficient token tree 700 starts at the node 703(i.e., when checkEob=0). The tokens ONE_EOB and TWO_EOB can be or cancorrespond to, respectively, the ONE_TOKEN and the TWO_TOKEN (i.e.,traversal of coefficient token tree 700 starting at the root node 701).Tree traversal of coefficient token tree 700 and checkEob are furtherdescribed below.

To encode or decode a token t[i] by using a binary arithmetic codingengine (such as by the entropy encoding stage 408 of FIG. 4), thecoefficient token tree 700 can be used. The coefficient token tree 700is traversed starting at the root node 701 (i.e., the node labeled A).Traversing the coefficient token tree generates a bit string (acodeword) that will be encoded into the bitstream using, for example,binary arithmetic coding. The bit string is a representation of thecurrent coefficient (i.e., the quantized transform coefficient beingencoded).

If a current coefficient is zero, and there are no more non-zero valuesfor the remaining transform coefficients, the token 702 (i.e., theEOB_TOKEN) is added into the bitstream. This is the case, for example,for the transform coefficient at scan order location 12 of FIG. 6. Onthe other hand, if the current coefficient is non-zero, or if there arenon-zero values among any remaining coefficients of the current block, a“1” bit is added to the codeword and traversal passes to the node 703(i.e., the node labeled B). At node B, the current coefficient is testedto see if it is equal to zero. If so, the left-hand branch is taken suchthat token 704 representing the value ZERO_TOKEN and a bit “0” is addedto the codeword. If not, a bit “1” is added to the codeword andtraversal passes to node C. At node C, the current coefficient is testedto see if it is greater than 1. If the current coefficient is equal to1, the left-hand branch is taken and token 706 representing the valueONE_TOKEN is added to the bitstream (i.e., a “0” bit is added to thecodeword). If the current coefficient is greater than 1, traversalpasses to node D to check the value of the current coefficient ascompared to the value 4. If the current coefficient is less than orequal to 4, traversal passes to node E and a “0” bit is added to thecodeword. At node E, a test for equality to the value “2” may be made.If true, token 706 representing the value “2” is added to the bitstream(i.e., a bit “0” is added to the codeword). Otherwise, at node F, thecurrent coefficient is tested against either the value “3” or the value“4” and either token 710 (i.e., bit “0” is added to the codeword) ortoken 712 (i.e., bit “1” is added to the codeword) to the bitstream asappropriate; and so on.

Essentially, a “0” bit is added to the codeword upon traversal to a leftchild node and a “1” bit is added to the codeword upon traversal to aright child node. A similar process is undertaken by a decoder whendecoding a codeword from a compressed bitstream. The decoder reads a bitfrom bit stream. If the bit is a “1,” the coefficient token tree istraversed to the right and if the bit is a “0,” the tree is traversed tothe left. The decoder reads then a next bit and repeats the processuntil traversal of the tree reaches a leaf node (i.e., a token). As anexample, to encode a token t[i]=THREE_TOKEN, starting from the root node(i.e., the root node 701), a binary string of 111010 is encoded. Asanother example, decoding the codeword 11100 results in the tokenTWO_TOKEN.

Note that the correspondence between “0” and “1” bits to left and rightchild nodes is merely a convention used to describe the encoding anddecoding processes. In some implementations, a different convention, forexample, in one where “1” corresponds to the left child node, and “0”corresponds to the right child node, can be used. As long as both theencoder and the decoder adopt the same convention, the processesdescribed herein apply.

Since an EOB_TOKEN is only possible after a nonzero coefficient, whenu[i−1] is zero (that is, when the quantized transform coefficient atlocation i−1 of the 1D array u is equal to zero), a decoder can inferthat the first bit must be 1. The first bit has to be 1 since, intraversing the tree, for a transform coefficient (e.g., transformcoefficient at the zig-zag scan order location 2 of FIG. 6) following azero transform coefficient (e.g., transform coefficient at the zig-zagscan order location 1 of FIG. 6), the traversal necessarily moves fromthe root node 701 to the node 703.

As such, a binary flag checkEob can be used to instruct the encoder andthe decoder to skip encoding and decoding the first bit leading from theroot node in the coefficient token tree 700. In effect, when the binaryflag checkEob is 0 (i.e., indicating that the root node should not bechecked), the root node 701 of the coefficient token tree 700 is skippedand the node 703 becomes the first node of coefficient token tree 700 tobe visited for traversal. That is, when the root node 701 is skipped,the encoder can skip encoding and the decoder can skip decoding and caninfer a first bit (i.e., a binary bit “1”) of the encoded string.

At the start of encoding or decoding a block, the binary flag checkEobcan be initialized to 1 (i.e., indicating that the root node should bechecked). The following steps illustrate an example process for decodingquantized transform coefficients in an N×N block.

At step 1, the binary flag checkEob is set to zero (i.e., checkEob=0)and an index i is also set to zero (i.e., i=0).

At step 2, a token t[i] is decoded by using either

-   -   1) the full coefficient token tree (i.e., starting at the root        node 701 of the coefficient token tree 700) if the binary flag        checkEob is equal to 1 or    -   2) using the partial tree (e.g., starting at the node 703) where        the EOB_TOKEN is skipped, if checkEob is equal to 0.

At step 3, If the token t[i]=EOB_TOKEN, then the quantized transformcoefficients u[i], . . . , u[N*N−1] are all to zero and the decodingprocess terminates; otherwise, extra bits can be decoded if necessary(i.e., when t[i] is not equal to the ZERO_TOKEN) and reconstruct u[i].

At step 4, the binary flag checkEob is set to 1 if u[i] is equal tozero, otherwise checkEob is set to 0. That is, checkEob can be set tothe value (u[i]!=0).

At step 5, the index i is incremented (i.e., i=i+1).

At step 6, the steps 2-5 are repeated until all quantized transformcoefficients have been decoded (i.e., until the index i=N*N) or untilthe EOB_TOKEN is decoded.

At step 2 above, decoding a token t[i] can include the steps ofdetermining a context ctx, determining a binary probability distribution(i.e., a model) from the context ctx, and using a boolean arithmeticcode to decode a path from the root node of the coefficient token tree700 to a leaf node by using the determined probability distributions.The context ctx can be determined using a method of context derivation.The method of context derivation can use one or more of the block size,plane type (i.e., luminance or chrominance), the position i, andpreviously decoded tokens t[0], . . . , t[i−1] to determine the contextctx. Other criteria can be used to determine the context ctx. The binaryprobability distribution can be determined for any internal node of thecoefficient token tree 700 starting from the root node 701 whencheckEOB=1 or from the node 703 when checkEOB=0.

In some encoding systems, the probability used to encode or decode atoken t[i] given a context ctx may be fixed and does not adapt in apicture (i.e., a frame). For example, the probability may be either adefault value that is defined for the given context ctx or theprobability may be coded (i.e., signaled) as part of the frame headerfor that frame. Coding the probability for every context in coding aframe can be costly. As such, an encoder may analyze, for each context,whether it is beneficial to code the context's associated probability inthe frame header and signal its decision to the decoder by using abinary flag. Furthermore, coding the probability for a context may useprediction to reduce cost (e.g., in bit rate) where the prediction maybe derived from the probability of the same context in a previouslydecoded frame.

FIG. 8 is a diagram of an example of a tree 800 for binarizing aquantized transform coefficient according to implementations of thisdisclosure. The tree 800 is a binary tree that can be used forbinarizing quantized transform coefficients in some video codingsystems. The tree 800 can be used by a video coding system that uses thesteps of binarization, context modelling, and binary arithmetic codingfor encoding and decoding of quantized transform coefficients. Theprocess may be referred to as context-adaptive binary arithmetic coding(CABAC). For example, to code a quantized transform coefficient x, thecoding system may perform the following steps. The quantized transformcoefficient x can be any of the coefficients (e.g., the coefficient 608)of the quantized transform block 604 of FIG. 6.

In the binarization step, a coefficient x is first binarized into abinary string by using the tree 800. The binarization process maybinarize the unsigned value of the coefficient x. For example,binarizing the coefficient 628 (i.e., the value −1), binarizes thevalue 1. This results in traversing the tree 800 and generating thebinary string 10. Each of the bits of the binary string 10 is referredto as a bin.

In the context derivation step, for each bin to be coded, a context isderived. A context can be derived from information such as one or moreof the block size, plane type (i.e., luminance or chrominance), blockposition of the coefficient x, and previously decoded coefficients(e.g., a left and/or above neighboring coefficients, if available).Other information can be used to derive the context.

In the binary arithmetic coding step, given a context, a bin is coded byusing, e.g., a binary arithmetic coding engine into a binary codewordtogether with a probability value associated with the context.

The steps of coding a transform coefficient can include a step that isreferred as context update. In the context update step, after a bin iscoded, the probability associated with the context is updated to reflectthe value of the bin.

Mixing of probability models is now described for coding (i.e., encodingor decoding) a sequence x^(n) of length n. For simplicity, two (2)models are used. However, this disclosure is not so limited and anynumber of models can be mixed.

Given a probability p(x^(n)) of a sequence of symbols x^(n), a goodentropy coding engine, such as a well-designed binary arithmetic codingengine, can produce from the probability p(x^(n)) a binary string oflength −log(p(x^(n))). As the length of the string is an integer number,“a binary string of length −log(p(x^(n)))” means a binary string havinga length that is the smallest integer that is greater than−log(p(x^(n))). Herein, when referring to a sequence of symbols, asuperscript of i refers to a sequence having a length of i symbols, anda subscript of i refers to the symbol at position i in the sequence. Forexample, x⁵ refers to a sequence of five (5) symbols, such as 11010;whereas x₅ refers to the symbol in the 5^(th) position, such as the last0 in the sequence 11010. As such the sequence x^(n) can be expressed asx^(n)=x₁x₂ . . . x_(n).

As used herein, probability values, such as the probability p(x^(i)) ofthe sub-sequence x^(i), can have either floating-point or fixed-pointrepresentations. Accordingly, operations applied to these values may useeither floating-point arithmetic or fixed-point arithmetic.

Given two probabilities p₁(x^(n)) and p₂(x^(n)) such thatp₁(x^(n))<p₂(x^(n)), the probability p₁(x^(n)) results in a codewordthat is no shorter than the probability p₂(x^(n)). That is, a smallerprobability typically produces a longer codeword than a largerprobability.

The underlying probability model from which symbols are emitted in videocoding is typically unknown and/or is likely too complex or too costlyto be fully described. As such, designing a good model for use inentropy coding can be a challenging problem in video coding. Forexample, a model that works well for one sequence may perform poorly foranother sequence. That is, given a first model and a second model, somesequences might compress better using the first model while othersequences might compress better using the second model.

In some video systems, it is possible to code (i.e., signal in anencoded bitstream) an optimal model for encoding a sequence. Forexample, given a sequence to be encoded, a video system may encode thesequence according to all or a subset of available models and thenselect the model that results in the best compression result. That is,it is possible to code the selection of a particular model among a setof more than one models for the sequence. In such a system, a two-passprocess may be, implicitly or explicitly, performed: a first pass todetermine the optimal model and a second to encode using the optimalmodel. A two-pass process may not be feasible in, e.g., real-timeapplications and other delay-sensitive applications.

As mentioned above, multiple models (i.e., models 1, . . . , M) may beavailable for entropy coding. For a sequence of symbols to be compressedwithout loss of information, mixing a finite number of models forarithmetic coding can be as good as selecting the best one model,asymptotically. This follows from the fact that the log function is aconcave function and that the −log function is a convex function.

From the foregoing, and for a finite sequence x^(n)=x₁x₂ . . . x_(n) oflength n, inequality (1) follows:−log(Σ_(k=1) ^(M) w _(k) p _(k)(x ^(n)))≤Σ_(k=1) ^(M) w _(k)(−log p_(k)(x ^(n)))  (1)

In the inequality (1), w_(k) denotes the weighting factor of the k^(th)model and p_(k)(x^(n)) denotes the joint probability of x^(n) given bymodel k. As described above, given a probability p_(k)(x^(n)) (i.e., theprobability given by model k of the sequence x^(n)) and x^(n) as theinput, an entropy coding engine can map x^(n) into a binary codeword oflength that is approximately equal to −log p_(k)(x^(n)).

From the inequality (1), it follows that taking the linear (i.e.,weighted) sum of the probabilities (i.e., Σ_(k=1) ^(M) w_(k)p_(k)(x^(n))) for the available models and then taking the logarithmof the linear sum is always less than or equal to taking the logarithmsof the probabilities (log p_(k)(x^(n))) of the models 1, . . . , M andthen performing a linear sum using the same weighting factors {w_(k)}.That is, the left-hand-side of the inequality is always less than orequal to the right-hand-side of the inequality.

It also follows from the inequality (1) that, given M models, it is moreadvantageous to mix the probabilities of the models 1, . . . , M beforeentropy coding a symbol. That is, it may be more advantageous to mix theprobabilities of multiple models before entropy coding than to choosemodels according to probabilities and using each model to individuallycode a sequence of bits. Mixing distinct models is likely to improvecompression performance (i.e., reduces compression rate) and is no worsethan selecting and coding the best model and then to code a sequenceusing the selected model.

The probability p_(k)(x^(n)) is a joint probability of the sequencex^(n). As coding x^(n) jointly can incur significant delay in processingand high computational complexity, mixing has found limited use, if atall, in video coding.

For any sub-sequence of length i of the sequence x^(n) where 1≤i≤n,probability p_(k)(x^(i)) denotes the probability of the subsequencex^(i) estimated by using model k, where k=1, 2. Using a correspondingweighting factor w_(k) for each model, the two models can be mixed usingequation (2):{tilde over (p)}(x ^(i))=w _(k) p _(k)(x ^(i)), for each i  (2)

In equation (2), {tilde over (p)}(x^(i)) is the mixed probability of thesub-sequence x^(i). As such, the mixing can produce partial (orintermediate) results for each sub-sequence x^(i). The sub-sequencex^(i) is x^(i)=x₁x₂x₃ . . . x₁. The first model (i.e., k=1) produces thesub-sequence probability p₁(x^(i)); and the second model (i.e., k=2)produces the sub-sequence probability p₂(x^(i)).

In an example, and as it may not be known, a priori, which model shouldhave the priority, a simple mixture can be used. For example, uniformweighting can be used. That is, the weight factors w_(k) can be chosensuch that w_(k)=½. As such, the equation (2) can be re-written as:{tilde over (p)}(x ^(i))=½Σ_(k=1) ² p _(k)(x ^(i)), for each i  (3)

The mixed probability {tilde over (p)}(x^(i)) is the probability of asub-sequence. However, arithmetic coding is performed on asymbol-by-symbol basis (i.e, not on sequences of symbols). As such, themixed probability {tilde over (p)}(x^(i)) cannot directly be used by forentropy coding. This can be addressed by converting the mixedprobability {tilde over (p)}(x^(i)) into a product of conditionalprobabilities as described below. It is also noted that the mixedprobability {tilde over (p)}(x^(i)) is itself a conditional probability:it is the probability of a symbol at position i having a certain valuegiven that the previous symbols result in the sub-sequence x^(i-1). Thatis, the mixed probability {tilde over (p)}(x^(i)) can be given byequation (4):{tilde over (p)}(x ^(i))={tilde over (p)}(x _(i) |x ^(i-1))  (4)

Using the elementary conditional probability formula P(A|B)=P(A ∩B)/P(B), where P(A ∩ B) is the probability of both events A and Boccurring, the equation (4) can be rewritten as equation (5):{tilde over (p)}(x ^(i))={tilde over (p)}(x _(i) |x ^(i-1))={tilde over(p)}(x _(i) ∩x ^(i-1))/{tilde over (p)}(x ^(i-1))={tilde over (p)}(x^(i))/{tilde over (p)}(x ^(i-1))  (5)

It is noted that the mixed probability of both x_(i) and x¹⁻¹ occurringis the same as the mixed probability of x^(i) alone because thesub-sequence x^(i) includes the sub-sequence x^(i-1) and has the symbolx_(i).

The equation (5) can be rewritten using the equation (3). That is, eachof the sub-sequence mixed probabilities (i.e., the numerator anddenominator) of equation (5) can be rewritten in terms of the modelprobabilities. The equation (5) can be rewritten as equation (6):

$\begin{matrix}{\begin{matrix}{{\overset{\sim}{p}\left( x^{i} \right)} = {\frac{1}{2}{\sum_{k = 1}^{2}{{p_{k}\left( x^{i} \right)}\text{/}\frac{1}{2}{\sum_{k = 1}^{2}{p_{k}\left( x^{i - 1} \right)}}}}}} \\{= {\frac{p_{1}\left( x^{i} \right)}{\sum_{k = 1}^{2}{p_{k}\left( x^{i - 1} \right)}} + \frac{p_{2}\left( x^{i} \right)}{\sum_{k = 1}^{2}{p_{k}\left( x^{i - 1} \right)}}}}\end{matrix}\quad} & (6)\end{matrix}$

Multiplying the first quantity and the second quantity of the equation(6) each by a factor equaling one (1)

$\left( {{i.e.},{\frac{p_{1}\left( {x^{i} - 1} \right)}{p_{1}\left( x^{i - 1} \right)}\mspace{14mu}{and}\mspace{14mu}\frac{p_{2}\left( {x^{i} - 1} \right)}{p_{2}\left( x^{i - 1} \right)}},} \right.$respectively), equation (7) is obtained:

$\begin{matrix}{{\overset{\sim}{p}\left( x^{i} \right)} = {{\frac{p_{1}\left( x^{i - 1} \right)}{\sum\limits_{k = 1}^{2}{p_{k}\left( x^{i - 1} \right)}}*\frac{p_{1}\left( x^{i} \right)}{p_{1}\left( x^{i - 1} \right)}} + {\frac{p_{2}\left( x^{i} \right)}{\sum\limits_{k = 1}^{2}{p_{k}\left( x^{i - 1} \right)}}*\frac{p_{2}\left( x^{i} \right)}{p_{2}\left( x^{i - 1} \right)}}}} & (7)\end{matrix}$

Equation (7) can be written as equation (8):{tilde over (p)}(x ^(i))=w _(i,1) *p ₁(x _(i) |x ^(i-1))+w _(i,2) *p ₂(x_(i) |x ^(i-1))  (8)

It is noteworthy that the conditional probabilities of p₁(x_(i)|x^(i|1))and p₂(x_(i)|x^(i-1)) are available as a result of the encoding of asequence up to the i^(th) symbol. These conditional probabilities areavailable since entropy encoding encodes one symbol at a time andgenerates a probability for the codeword (up to and including x_(i))with every symbol. In implementations according to this disclose, theconditional probabilities are mixed and the sequence is then encoded (ordecoded) using the mixed probability (i.e., {tilde over (p)}(x^(i))).

In equation (8), w_(i,1) and w_(i,2) are weights that are respectivelyequal to

${\frac{p_{1}\left( x^{i - 1} \right)}{\sum\limits_{k = 1}^{2}{p_{k}\left( x^{i - 1} \right)}}\mspace{14mu}{and}\mspace{14mu}\frac{p_{2}\left( x^{i - 1} \right)}{\sum\limits_{k = 1}^{2}{p_{k}\left( x^{i - 1} \right)}}},$and p₁(x_(i)|x^(i-1)) and p₂(x_(i)|x^(i-1)) are respectively equal to

$\frac{p_{1}\left( x^{i} \right)}{p_{k}\left( x^{i - 1} \right)}\mspace{14mu}{and}\mspace{14mu}{\frac{p_{2}\left( x^{i} \right)}{p_{k}\left( x^{i - 1} \right)}.}$As such, the mixed probability {tilde over (p)}(x^(i)) is now expressedas a linear combination of the conditional probability of the firstmodel (i.e., p₁(x_(i)|x^(i-1))) and the conditional probability of thesecond model (i.e., p₂(x_(i)|x^(i-1))) where each of the conditionalprobabilities is multiplied by a respective weighting factor.

When the joint distributions are mixed using the equation (3), uniformweighting factors (i.e., ½) were used. However, when conditionalprobabilities are used mixed (as in the equation (8)), the weighting(i.e., w_(i,1) for the first model and w_(i,2) for the second model) mayno longer be uniform. The weight w_(i,1) for the conditional probabilityof the first model is equal to the joint probability of x^(i-1) given bythe first model divided by the sum of the joint probability of x^(i-1)given by the first model and the joint probability of x^(i-1) given bythe second model. Similarly for weight w_(i,2). In equation (8), for thesub-sequence x^(i-1), the first model provides a first probability andthe second model provides a second probability and the weighting factorfor the conditional probability of x_(i) given x^(i-1) is equal to theprobability given by each of the first model and the second modeldivided by the sum of the joint probabilities given by both models. Thatis, in the mixing of the conditional probabilities, if, for example, thefirst model provides a higher probability for the sub-sequence x^(i-1),then the first model ends up having a higher weighting factor (i.e.,weight w_(i,1)) than that of the second model.

The joint probabilities are real numbers and the calculating of theweights w_(i,1) and w_(i,2) involves the division of real numbers. Assuch, the computing of the weights w_(i,1) and w_(i,2) may be complexand expensive. It is desirable to approximate the weights w_(i,1) andw_(i,2) with fixed-point representations such that, for example, theexact number of bits to represent each of the weights can be known andsuch that division operations can be avoided.

As described above, there is a correlation and/or relationship betweenthe probability of a codeword and the length, in bits, of the codewordgenerate using the probability. Namely, the length (i.e., number ofbits) of the codeword is given by −log₂(p). The lengths of the codewordsgenerated by each model can be used to approximate the weights w_(i,1)and w_(i,2). That is, −log(p_(k)(x^(i-1))) can be approximated by thecodeword length l_(k)(x^(i-1)) in bits resulting from using model k,k=1,2, to encode x^(i-1). As such, the weight w_(i,1) (and for theweight w_(i,2)) can be approximated using equation (9):

$\begin{matrix}{w_{i,1} = {{\frac{p_{k}\left( x^{i - 1} \right)}{\sum\limits_{j = 1}^{2}{p_{j}\left( x^{i - 1} \right)}} \approx \frac{2^{- {l_{1}{(x^{i - 1})}}}}{\sum\limits_{j = 1}^{2}2^{- {l_{j}{(x^{i - 1})}}}}} = \frac{1}{1 + 2^{{l_{1}{({x^{i} - 1})}} - {l_{2}{({x^{i} - 1})}}}}}} & (9)\end{matrix}$

When l₂(i−1) is equal to l₁(i−1), then it follows thatw_(i,1)=w_(i,2)=0.5. Assuming, without losing generality, that l₁(i−1)is smaller than l₂(i−1), then the equation (9) can result by expandingthe denominator and then eliminating 2^(−l) ¹ (x^(i-1)) from thedenominator and numerator.

To determine a length l_(k)(x^(i)) according to a model k of asub-sequence of length i, a hypothetical encoding process can be used. Ahypothetical encoding process is a process that carries out the codingsteps but does not generate actual codewords or output bits into anencoded bitstream. Since the purpose is to estimate l_(k)(x^(i)), whichare interpreted in some applications as a bitrate (or a simply rate), ahypothetical encoding process may be regarded or called a rateestimation process. The hypothetical encoding process, using aprobability model, computes or estimates the codeword length for asequence. The codeword length may be determined (i.e., measured) with orwithout generating a codeword. For example, at time instance i, codingthe sequence x^(i-1) using a first model generates a codeword of lengthl₁(i−1) and using a second model generates a codeword of length l₂(i−1).In an example, multiple hypothetical encoders can be available andexecuting in parallel. For example, a standard rate estimator for anarithmetic encoder can be available for each model. Each rate estimatorcan provide (or, can be used to provide) an estimate of the length ofthe codeword that may be produced by the encoder for a sub-sequencegiven a model.

Given two competing models at a time instance i, if the first modelprovides less bits than the second model, then the weight assigned(using equation 9) to the first model will be greater than the weightassigned to the second model for the sequence up to the symbol atposition x_(i−1). Eventually (i.e., when encoding the sequence x^(n) iscompleted using the mixed probability), the winning model (i.e., themodel with higher weight) is the model that produces less bits, which isthe desired result of compression.

The weight w_(i,1) is approximated (in equation (9)) using a power of 2and, as such, can be efficiently computed.

The weight w_(i,1) can be further simplified. The right-hand-side of theequation (9) is of the form 1/(1−r) where r=−2^(l) ¹(x^(i-1))−l₂(x^(i-1)). This can be recognized as a geometric seriesgiven by 1+r+r²+ . . . with a common ratio r=−2^(l) ¹(x^(i-1))−l₂(x^(i-1)). As such, The weight w_(i,1) can be approximatedusing equation (10):w _(i,1)≈Σ_(j=0) ^(∞)(−2^(l) ² (x ^(i-1))−l ₂(x ^(i-1)))^(j)  (10)

As such, w_(i,1)*p₁(x_(i)|x^(i-1)) of the equation (8) can be rewrittenas in equation (11):w _(i,1) p ₁(x ^(i) |x ^(i-1))=Σ_(j=0) ^(∞)(−2^(l) ¹ ^((x) ^(i-1) ^()−l)² ^((x) ^(i-1) ⁾)^(j) p ₁(x _(i) |x ^(i-1))=Σ_(j=0) ^(∞)(−1)^(j)2^(j[l)¹ ^((x) ^(i-1) ^()−l) ² ^((x) ^(i-1) ^()]) p ₁(x _(i) |x ^(i-1))  (11)

In equation (11), 2^(j[1) ¹ ^((x) ^(i-1) ^()−l) ² ^((x) ^(i-1)^()])p₁(x_(i)|x^(i-1)) can be efficiently computed by using shifts incases where p₁(x_(i)|x^(i-1)) has a fixed-point representation.Moreover, when p₁(x_(i)|x^(i-1)) has a fixed-point representation, thenthe infinite sum in equation (11) can be truncated into a sum of afinite number of terms. For example, when p₁(x_(i)|x^(i-1)) has an 8-bitrepresentation, then the sum can be truncated to keep only the firsteight (8) terms Σ_(j=0) ⁷(−1)^(j)2^(j[l) ¹ ^((x) ^(i-1) ^()−l) ² ^((x)^(i-1) ^()])p₁(x_(i)|x^(i-1)) since for any j≥8, 2^(j[l) ¹ ^((x) ^(i-1)^()−l) ² ^((x) ^(i-1) ^()])p₁(x_(i)|x^(i-1))==0 whenl₁(x^(i-1))−l₂(x^(i-1))≤−1 (that is, when they differ by at least onebit). When, l₁(x^(i-1))−l₂(x^(i-1))<−1, (that is, when they differ bymore than one bit), 2^(j[l) ¹ ^((x) ^(i-1) ^()−l) ² ^((x) ^(i-1) ^()]) p₁(x_(i)|x^(i-1))=0 for any j≥j* where j*<8. As such, only the first j*terms are needed to compute w_(i,1)p₁(x^(i)|x^(i-1)).

The weight w_(i,2) can be computed using equation (12):

$\begin{matrix}{{w_{i,2} \approx \frac{2^{- {l_{1}{(x^{i - 1})}}}}{\sum\limits_{j = 1}^{2}2^{- {l_{j}{(x^{i - 1})}}}}} = {\frac{2^{{l_{1}{({x^{i} - 1})}} - {l_{2}{({x^{i} - 1})}}}}{1 + 2^{{l_{1}{({x^{i} - 1})}} - {l_{2}{({x^{i} - 1})}}}} = {2^{{l_{1}{({x^{i} - 1})}} - {l_{2}{({x^{i} - 1})}}}{\sum\limits_{j = 0}^{\infty}{\left( {- 2^{{l_{1}{({x^{i} - 1})}} - {l_{2}{({x^{i} - 1})}}}} \right)j}}}}} & (12)\end{matrix}$

The quantity w_(i,2)*p₂(x_(i)|x^(i-1)) of equation (8) can be computedusing equation (13):w _(i,2) p ₂(x ^(i) |x ^(i-1))=2^(l) ² ^((x) ^(i-1) ^()−l) ² ^((x)^(i-1) ⁾Σ_(j=0) ^(∞)(−2^(l) ¹ ^((x) ^(i-1) ^()−l) ² ^((x) ^(i-1) ⁾)^(j)p ₂(x _(i) |x ^(i-1))=Σ_(j=0) ^(∞)(−1)^(j)2^((j+1)[l) ¹ ^((x) ^(i-1)^()−l) ² ^((x) ^(i-1) ^()]) p ₂(x ₁ |x ^(i-1))  (13)

As in equation (11), the right hand side of equation (13) can besimplified by truncating the infinite sum into a finite sum whenp₂(x_(i)|x^(i-1)) has a fixed-point representation.

As described above, mixing of joint probabilities of models can usesimple uniform mixing as it may not be known a priori which modelprovides better compression. The uniform mixing of the jointprobabilities uses conditional probabilities and results in theselection of a winning model (i.e., a model with higher weighting).

FIG. 9 is a flowchart diagram of a process 900 for encoding a sequenceof symbols according to an implementation of this disclosure. Theprocess 900 can receive a sequence of symbols of size n. The sequencecan be denoted by x^(n). Receive, can mean generate, determine, or inany way receive. In an example, the sequence of symbols can represent aquantized transform coefficient such as one received at the entropyencoding stage 408 from the quantization stage 406 of FIG. 4. In anexample, the sequence of symbols can be a token such as a tokendescribed with respect to FIG. 7. In an example, the sequence of symbolscan be a binarized value such as a binarized value described withrespect to FIG. 8. The sequence of symbols can be any sequence ofsymbols that is encoded based on a probability model.

The process 900 can be implemented in an encoder such as the encoder 400of FIG. 4. The process 900 can be implemented, for example, as asoftware program that can be executed by computing devices such astransmitting station 102. The software program can includemachine-readable instructions that can be stored in a memory such as thememory 204 or the secondary storage 214, and that can be executed by aprocessor, such as CPU 202, to cause the computing device to perform theprocess 900. In at least some implementations, the process 900 can beperformed in whole or in part by the entropy encoding stage 408 of theencoder 400 of FIG. 4.

The process 900 uses at least two probability models to encode thesequence of symbols x^(n). The process 900 can use any number ofprobability models. However, for simplicity only two (2) models (i.e., afirst model and a second model) are used to illustrate the process 900.The process 900 encodes each of the symbols of the sequence the symbolsby mixing the probabilities of the first model and the second model.

At 902, the process 900 initializes a counter i to 0, a firstsub-sequence length (i.e., first length l₁) to 0, and a secondsub-sequence length (i.e., second length l₂) to 0. The counter i is usedfor each symbol of the sequence x^(n). The first length l₁ and thesecond length l₂ are as described above. That is, the first length l₁and the second length l₂ can correspond, respectively, to the lengths ofa codewords generated by arithmetic coding engines using the first modeland the second model.

At 904, the process 900 determines the conditional probabilitiesp₁(x_(i)|x^(i-1)) and p₂(x_(i)|x^(i-1)) as described above. Theconditional probability p₁(x_(i)|x^(i-1)) is the conditional probabilityof the symbol at position i of the sequence of symbols given theprobability of the subsequence x^(i-1) (i.e., the sub-sequence up to andexcluding the symbol x_(i)). Similarly for p₂(x_(i)|x^(i-1)).

At 906, the process 900 computes the mixed probability {tilde over(p)}(x_(i)|x^(i-1)) for the symbol x_(i). The process 900 can determinethe mixed probability described in equation (4) above. The process 900can compute the mixed probability using the equations 8, 11, and 13. At908, the process 900 encodes the symbol x_(i) using the computed mixedconditional probability.

At 910, the process 900 updates the first length l₁ and the secondlength l₂. As described above, hypothetical arithmetic encoders can beused at 910. The first length l₁ is updated to include the additionalcodeword length (i.e., bits) added to the hypothetical codeword added bythe first model when encoding the symbol x_(i). The second length l₂ isupdated to include the additional codeword length (i.e., bits) added tothe hypothetical codeword added by the second model when encoding thesymbol x_(i). The process 900, updates the first length l₁ and thesecond length l₂ using, respectively, l₁=l₁−log(p₁(x_(i)|x^(i-1))) andl₂=l₂−log(p₂(x_(i)|x^(i-1))). In an implementation, the values−log(p₁(x_(i)|x^(i-1))) and −log(p₂(x_(i)|x^(i-1))) can be computedand/or approximated by using a lookup table. Note that the probabilitiesp₁(x_(i)|x^(i-1)) and p₂(x_(i)|x^(i-1)) are probabilities between zero(0) and one (1). If p_(i)(x_(i)|x^(i-1)) and p₂(x_(i)|x^(i-1)) are eachrepresented and/or approximated using an 8-bit integer (e.g., they havefixed-point representations), then both −log(p₁(x_(i)|x^(i-1))) and−log(p₂(x_(i)|x^(i-1))) can be estimated by using a lookup table thattakes 8-bit integers as inputs, where each input corresponds to aprobability value. In general, the size of the lookup table depends uponthe width of the fixed point representation of p₁(x₁|x^(i-1)) andp₂(x_(i)|x^(i-1)). That is, the larger the width, the higher theprecision in estimating −log(p₁(x₁|x^(i-1))) and log(p₂(x₁|x^(i-1))).

At 912, the counter i is incremented so that the next symbol x_(i+1) isprocessed. At 914, if all the symbols have been processed (i.e., i=n+1),then the process terminates at 916. Otherwise, the process returns to904 to process the next symbol.

FIG. 10 is a flowchart diagram of a process 1000 for decoding a sequenceof symbols according to an implementation of this disclosure. Theprocess 1000 can be implemented in a decoder such as the decoder 500.The process 1000 can be implemented by a receiving station. The process900 can be implemented, for example, as a software program that can beexecuted by computing devices. The software program can includemachine-readable instructions that can be stored in a memory such as thememory 204 or the secondary storage 214, and that can be executed by aprocessor, such as CPU 202, to cause the computing device to perform theprocess 900. The process 900 can be implemented using specializedhardware or firmware. Some computing devices can have multiple memories,multiple processors, or both. The steps or operations of the process1000 can be distributed using different processors, memories, or both.Use of the terms “processor” or “memory” in the singular encompassescomputing devices that have one processor or one memory as well asdevices that have multiple processors or multiple memories that can beused in the performance of some or all of the recited steps.

The process 1000 can be used to decode a sequence of symbols from anencoded bitstream. For example, the process 1000 can receive an encodedbitstream, such the compressed bitstream 420 of FIG. 5. The process 1000can include steps similar to the steps 902-906 and 910-916 as theprocess 900. Descriptions of the similar steps are omitted. Instead ofthe step 908, the process 1000 includes the step 1002. At the step 1002,the process 1000 decodes, from an encoded bitstream, the symbol x_(i)using the computed mixed conditional probability (i.e., {tilde over(p)}(x_(i)|x^(i-1))).

In some implementations of the processes 900 or 1000, step 906 may beperformed every k>1 steps to further save (e.g., reduce) computationalcomplexity or to improve throughput. Throughput can be measured in thenumber symbols processed (coded or decoded) in one clock cycle. Forexample, when k=2, step 906 may be performed only when i is odd or even,but not both. In another implementation of the processes 900 or 1000,step 906 may be performed at a predefined subset of all possible indicesof i.

The foregoing described the use of uniform weighting of the models.However, implementations according to this disclosure can usenon-uniform prior weights. In non-uniform weighting using M number ofmodels, at least some of the weights w_(k) can be set to values that arenot equal to 1/M (i.e., w_(k)≠1/M).

For simplicity, the foregoing (e.g., the processes 900 and 1000)describes the use of two models: a first model and a second model.However, implementations according to this disclosure can be extended toany number of models. For example, for a number of model M≥2, andassuming uniform weighting factors w_(k) (i.e., w_(k)=1/M), then theweights w_(i,k) can be approximated using formula (14):

$\begin{matrix}{w_{i,k} \approx \frac{w_{k}2^{- {l_{k}{({x^{i} - 1})}}}}{\sum\limits_{j = 1}^{M}{w_{j}2^{- {l_{j}{(x^{i - 1})}}}}}} & (14)\end{matrix}$

In formula 14, l_(k) (x^(i-1)) denotes the codeword length, in bits,resulting from using model k, 1≤k≤M, to encode the sub-sequence x^(i-1).

In the case where more than two (2) models are mixed, a binary tree canbe used to compute (i.e., determine, generate, etc.) the conditionalprobabilities. That is, the factors w_(i,k)p_(k)(x_(i)|x^(i-1)) ofequation (8) can be recursively computed using the above-describedprocesses. Recursively computing means combining the probabilities oftwo (2) models at a time to produce intermediate conditionalprobabilities. The intermediate conditional probabilities are thencombined, two at a time. In the case where the number of models M is apower of 2 (i.e., M=2^(m)), the factors w_(i,k)p_(k)(x_(i)|x^(i-1)) ofequation (8) can be recursively computed by applying the above describedprocesses on a full binary tree such as described with respect to FIG.11.

FIG. 11 is a diagram of an example of a binary tree 1100 of conditionalprobabilities according to an implementation of this disclosure. In thebinary tree 1100, eight (8) models are mixed. The probabilities of theeight models are p_1 to p_8. Every two probabilities are first mixed.For example, the probabilities 1102 and 1104 are mixed as describedabove to generate intermediate conditional probability 1106, which isthen combined with the intermediate conditional probability 1108 toproduce intermediate conditional probability 1110, and so on until afinal conditional probability 1112 is computed. The final conditionalprobability 1112 can be used for encoding and/or decoding. For example,the final conditional probability 1112 can be used at 908 of the process900 and/or at 1002 of the process 1000.

The process described with respect to FIG. 11 can be used in situationswhere, for example, some models are known to be more useful than othermodels. In the case where some models are known to be more useful thanothers, uniform weighting may be undesirable. In order to assign moreweight to one model, the model can be replicated in the tree.

Referring to FIG. 11 as an example, the models p_1-p_6 and p_8 may bedistinct and p_6 is known to be more useful than the other models. Asp_6 is more useful, p_6 can be replicated in the tree: p_7 is aduplicate of p_6. As such, the model with probability p_6 is assignedtwice the weight in the mixing for entropy encoding.

As another example, suppose, for example, there are two models, model Aand model B, and the prior weights for the two models are (¼, ¾).Implementations according to this disclosure, can expand the model setto a set of 4 models, where the first model corresponds to the model A,the remaining three models correspond to the model B, and the prior forthe four models is (¼, ¼, ¼, ¼).

In the foregoing, stationary sources are described. A stationary sourcemeans that the mixing for the symbol x_(i) uses all the history of thesub-sequence x⁻¹ to determine w_(i,k). As such, the statistics do notchange over the source of the coding process. However, in the caseswhere the sources may be non-stationary, implementations according thisdisclosure can adapt to the local statistics for better compressionperformance using a sliding window. The sliding window as length L ofbits indicating the number of previous bits (i.e., the probabilities ofthe number of the previous bits) to be used in the mixing process. Thatis, the sliding window represents how far back into the sequence toremember: only symbols inside the sliding window are used to estimatethe weighting factors. More specifically, only the probabilities ofthose symbols inside the sliding window are used to estimate theweighting factors.

As such, instead of using {tilde over (p)}(x_(i)|x^(i-1)) to code x_(i),{tilde over (p)}(x_(i)|x_(i−L) . . . x_(i−1)) where the length L≥1 isthe length of the sliding window and where x_(i−L) . . . x_(i−1) is thesub-sequence starting at bit i−L and ending at bit i−1. When the lengthL is known, a process according to this disclosure can perform thefollowing steps for two models:

At step 1, initialize i=1, l₁=0, l₂=0. The step 1 can be as describedwith respect to 902 of FIG. 9. At step 1, the process also initializesl_(1,−L)=0, and l_(2,−L)=0.

At step 2, the process computes p₁(x_(i)|x_(i−L) . . . x_(i−1)) andp₂(x_(i)|x_(i−L) . . . x_(i−1)) according to the first model and thesecond model.

At step 3, the process computes the mixed probability {tilde over(p)}(x_(i)|x_(i−L) . . . x_(i−1)) according to the equations 15 and 16:

$\begin{matrix}{{\overset{\sim}{p}\left( {x_{i}\text{|}x_{i - L}\ldots\; x_{i - 1}} \right)} = {{w_{i,1}{p_{1}\left( {x_{i}\text{|}x_{i - L}\ldots\; x_{i - 1}} \right)}} + {w_{i,2}{p_{2}\left( {x_{i}\text{|}x_{i - L}\ldots\; x_{i - 1}} \right)}}}} & (15) \\{\mspace{79mu}{{w_{i,k} \approx \frac{2^{{- {l_{1}{({x^{i} - 1})}}} + {l_{1}{({x^{i} - L - 1})}}}}{\sum\limits_{j = 1}^{2}2^{{- {l_{j}{({x^{i} - 1})}}} + {l_{j}{({x^{i} - L - 1})}}}}},{k = 1},2}} & (16)\end{matrix}$

At step 4, the process encodes (when implemented by an encoder) ordecodes (when implemented by a decoder) x_(i) by using {tilde over(p)}(x_(i)|x_(i−L) . . . x_(i−1)).

At step 5, the process updates l₁ to l₁=l₁−log p₁(x_(i)|x_(i−L) . . .x_(i−1)) and updates l₂ to l₂=l₂−log p₂(x_(i)|x_(i−L) . . . x_(i−1)). Ifthe process is encoding/decoding outside the window (i.e., i>L), thenthe process updates l_(1,−L)=l_(1,−L)−log p₁(x_(i−L)|x_(i−2L) . . .x_(i−L−1)) and l_(2,−L)=l_(2,−L)−log p₂(x_(i−L)|x_(i−2L) . . .x_(i−L−1)).

At step 6, i is increased by 1 (i.e., i=i+1).

At step 7, the process repeats the steps 2-6 until all the bits of thesequence x^(n) are processed (i.e., i=n+1).

In the sliding window described above,l₁(x^(i-1))−l₁(x^(i-L-1))=l₁−l_(1,−L) andl₂(x^(i-1))−l₂(x^(i-L-1))=l₂−l_(2,−L). As such,l₁(x^(i-1))−l₁(x^(i-L-1)) can be regarded as the codeword lengthproduced by using the first model to code x_(i−L) . . . x_(i−1) and l₂(x^(i-1))−l₂(x^(i-L-1)) can be regarded as the codeword length producedby using the second model to code x_(i−L) . . . x_(i−1).

FIG. 12 is a flowchart diagram of a process 1200 for entropy coding asequence of symbols according to an implementation of this disclosure.The sequence can be as described above for sequences x^(n). The process1200 can be implemented by an encoder or a decoder. When implemented byan encoder, “coding” means encoding in an encoded bitstream, such as thecompressed bitstream 420 of FIG. 4. When implemented by a decoder,“coding” means decoding from a encoded bitstream, such as the compressedbitstream 420 of FIG. 5.

When encoded by an encoder, the process 1200 can receive the sequence ofsymbols from a quantization step, such as the quantization stage 406 ofFIG. 4. In another example, the process 1200 can receive a value to beencoded (e.g., a quantized transform coefficient) and generates thesequence of symbols from the received value.

At 1202, the process 1200 selects models to be mixed. The models caninclude a first model and a second model. As used in this disclosure,“select” means to identify, construct, determine, specify or otherselect in any manner whatsoever.

For at least a symbol (e.g., x_(i)), at a position (e.g., i) of thesymbols, the process 1200 performs steps including the steps 1204-1208to determine a mixed probability using the first model and the secondmodel. The steps 1204-1208 can be performed for all symbols of thesequence of symbols.

At 1204, the process 1200 determines, using the first model, a firstconditional probability for encoding the symbol. The first conditionalprobability is the conditional probability of the symbol given asub-sequence of the sequence. In an example, the sub-sequence of thesequence can mean the sub-sequence x^(i-1). In another example, whereina sliding window is being used, the sub-sequence of the sequenceconsists of a predetermined number of symbols of the sequence before theposition. The predetermined number of symbols can be as described withrespect to the sliding window length L. As such the sub-sequence of thesequence can be the sub-sequence x_(i−L) . . . x_(i−1). At 1206, theprocess 1200 determines, using the second model, a second conditionalprobability for encoding the symbol. The second conditional probabilityis a conditional probability of the symbol given the sub-sequence asdescribed with respect to 1204.

At 1208, the process 1200 determines, using the first conditionalprobability and the second conditional probability, a mixed probabilityfor encoding the symbol. The mixed probability can be as described withrespect to 906 of FIG. 9. The first conditional probability and thesecond conditional probability can be combined using a linearcombination that uses a first weight and a second weight. In animplementation, at least the first weight can be determined (i.e.,approximated) using a hypothetical arithmetic coding to determine alength for encoding a sub-sequence of the sequence up to the symbol. Thefirst weight can be determined using the length. In an example,determining a weight (e.g., the first weight and/or the second weight)can include determining a rate resulting from encoding a sub-sequence ofthe sequence up to the symbol and determining the first weight using thedetermined rate. In an example, the rate can be determined using a rateestimator. In an example, the rate estimator can be a hypotheticalarithmetic encoder. In an example, determining the rate can includelooking up a table (e.g., a lookup table) with inputs as probabilityvalues.

At 1210, the process 1200 codes the symbol using the mixed probabilityas described, for example, with respect to the 908 (when implemented byan encoder) and 1002 (when implemented by a decoder).

In an implementation of the process 1200, the models can include a thirdmodel and a fourth model and determining the mixed probability using thefirst model and the second model can include mixing the first model andthe second model to generate a first intermediate conditionalprobability, mixing the third model and the fourth model to generate asecond intermediate conditional probability, and mixing the firstintermediate conditional probability and the second intermediateconditional probability to generate a conditional probability to be usedfor encoding the symbol. In an implementation, the first model and thefourth model are a same model.

A technique known as context-tree weighting (CTW) is a lossless datacompression algorithm that uses mixing. To code a binary sequence x^(n)of length n, CTW estimates a probability function p(x^(n)) as a linearmixture of 2^(K) probability functions p_(i)(x^(n)), each of which isestimated by assuming a finite memory binary tree source and has thesame weighting factor. Contrastingly, implementations according to thisdisclosure can work with any models. Furthermore, the symbol-by-symbolweighting factor computation described herein can use length functionsto approximate probabilities of sub-sequences, which is much simplifiedin comparison to existing solutions that maintain and compute jointprobabilities.

As mentioned above, a key design challenge or problem in contextmodeling is to balance between the two conflicting objectives of 1)improving compression performance by adding more contexts and 2)reducing the overhead cost associated with contexts.

Using the coefficient token tree 700 as a non-limiting illustration, amathematical analysis of the impact of the number of contexts in acoding system and the relationship between the number of contexts andthe coding performance is now given.

For a context c, let P_(c)=(p_(c,0), . . . , p_(c,11)) denote theprobability distribution obtained from the context, where p_(c,i)denotes the probability of token i for i=0, . . . , 10 (i.e., the tokenslisted in Table I), and p_(c,11) denotes the probability of theEOB_TOKEN. For convenience, the EOB_TOKEN is referred to as token 11.Suppose now that, in a frame, the context c appears and is used n_(c)times. A context “appears” when, for example, the conditions associatedwith context are met and/or are available for the frame. The context is“used” when, for example, the probability distribution associated withthe context is used in the coding of at least one block of the frame.Let Q_(c)=(q_(c,0), . . . , q_(c,11)) denote the empirical (i.e.,observed) distribution of the coefficient tokens under context c. Thatis, q_(c,i) denotes the number of times that token i appears in theframe under the context c and can be given by

$q_{c,i} = \frac{n_{c,i}}{n_{c}}$(i.e., the number of times that the token i appeared in the frame underthe context c divided by the number of times that the context cappeared). The probability distributions P_(c) and Q_(c) can be referredto, respectively, as the coding distribution and the actual distributionused to code a tokens.

Given the coding distribution P_(c), the compression performanceachievable by using arithmetic coding can be given by

$\sum\limits_{i = 0}^{11}{n_{c,i}\log{\frac{1}{p_{c,i}}.}}$Using the actual distribution

$Q_{c},{q_{c,i} = \frac{n_{c,i}}{n_{c}}},$and the law of logarithms log(fraction)=log(numerator)−log(denominator),the achievable compression performance can be reduced as in equation(17)

$\begin{matrix}{{\sum\limits_{i = 0}^{11}{n_{c,i}\log\frac{1}{p_{c,i}}}} = {{n_{c}{\sum\limits_{i = 0}^{11}{q_{c,i}\log\frac{1}{p_{c,i}}}}} = {{n_{c}{\sum\limits_{i = 0}^{11}{q_{c,i}\log\frac{1}{q_{c,i}}}}} + {n_{c}{\sum\limits_{i = 0}^{11}{q_{c,i}\log\frac{q_{c,i}}{p_{c,i}}}}}}}} & (17)\end{matrix}$

The first term

$\sum\limits_{i = 0}^{11}{q_{c,i}\log\frac{1}{q_{c,i}}}$of the right hand side of equation (17) can be recognized as the entropyH(Q_(c)) of the actual distribution Q_(c). The second term

$\sum\limits_{i = 0}^{11}{q_{c,i}\log\frac{q_{c,i}}{p_{c,i}}}$of the right hand side of equation (1) can be recognized as the relativeentropy or the Kullback-Leibler (KL) divergence, defined on the samealphabet (e.g., the tokens of the coefficient token tree 700), betweenthe distributions P_(c) and Q_(c). The KL divergence can be denotedD(Q_(c)∥P_(c)). As such, the compression performance achievable by usingarithmetic coding can be rewritten using equation (18):

$\begin{matrix}{{\sum\limits_{i = 0}^{11}{n_{c,i}\log\frac{1}{p_{c,i}}}} = {{n_{c}{H\left( Q_{c} \right)}} + {n_{c}{D\left( {Q_{c}\left. P_{c} \right)} \right.}}}} & (18)\end{matrix}$

The second term (i.e., D(Q_(c)∥P_(c))) is indicative of the loss incompression performance that results when, instead of using the mostoptimal probability distribution (i.e., the actual distribution Q_(c)),another less optimal probability distribution (i.e., the codingdistribution P_(c)) is used. When there are differences between theactual and coding probability distributions, a compression performanceloss results. The loss grows linearly with the sequence length beingencoded.

The equation (18) can be used as the basis for designing compressionalgorithms. That is, a compression algorithm is analyzed using theequation (18). The design of context modeling directly affects thecompression performance. As such, a good design of context modeling(i.e., optimal selection of contexts) results in a good compressionalgorithm. With optimal context modeling, the first term H(Q_(c)) andthe second term D(Q_(c)∥P_(c)) are minimized.

Both terms of the equation (18) (i.e., n_(c)H(Q_(c)) andn_(c)D(Q_(c)∥P_(c))) grow linearly with the number n_(c) of times thatthe context c appears. It can be appreciated that, in order to improvethe compression performance of entropy coding, the two terms, H(Q_(c))and D(Q_(c)∥P_(c)) of equation (18) should be made as small as possible.

As the second term D(Q_(c)∥P_(c)) is always a non-negative value, andD(Q_(c)∥P_(c))=0 if and only if the actual and coding distributions areequivalent (i.e., Q_(c)≡P_(c)), the first term (i.e., n_(c)H(Q_(c))) isthe absolute theoretical lower bound for any compression algorithm. Saidanother way, given a sequence n_(c) with a probability distributionQ_(c), the best possible compression is given by n_(c)H(Q_(c)) and noother probability distribution can provide better compression.

Accordingly, in order to achieve good compression performance, it isdesirable that the coding distribution P_(c) be as close as possible tothe actual distribution Q_(c). For a same context, the actualdistributions change from one frame to another. As such, the codingdistribution P_(c) should be adapted to the actual distribution Q_(c) ofa given frame to be decoded. As the frame is yet to be decoded, adecoder cannot know how to adjust the coding distribution P_(c).

Instead, an adjusted coding distribution P_(c) can be signaled by anencoder. For example, an encoder can encode the adjusted codingdistribution P_(c) in the frame header of the frame to be decoded.Encoding the adjusted coding distribution P_(c) can mean that theencoder encodes the token probabilities p_(c,i) of the codingdistribution P_(c).

Suppose that the cost, in bits (or more accurately, in partial bits), ofencoding the token probabilities p_(c,i) is lower-bounded by a bit costε>0. That is, bit cost c is the smallest number of bits required toencode a token probability p_(c,i).

For a context c, the total bit cost, amortized over the frame, of codingthe coding distribution P_(c), which includes the probabilities of the12 tokens of the Table I and the EOB_TOKEN, can be given by 11ε/n_(c).The total bit cost is inversely proportional to n_(c) and grows linearlywith the alphabet size (e.g., the number of tokens). Since the codingdistribution P_(c) is a probability distribution (i.e., Σ_(i=0)¹¹p_(c,i)=1), its degree of freedom is equal to the alphabet size (e.g.,12 tokens) minus one (1). Hence, the 11, instead of 12 (corresponding tothe number of tokens) in the bit cost 11ε/n_(c).

Let C denote the set of all contexts used in transform coefficientcoding. To transmit a coding probability distribution for a context c ofthe set C to a decoder (the decoder can use to decode tokenscorresponding to, e.g., nodes of the coefficient token tree 700 of FIG.7), a probability corresponding to each token of the coefficient tokentree 700 and the EOB_TOKEN may be encoded. As there are 12 tokens, thetotal bit cost of coding probability distributions obtained fromcontexts in the set C, amortized over a frame, is given by equation(19):

$\begin{matrix}{{{Total}\mspace{14mu}{bit}\mspace{14mu}{cost}} = {\frac{11{C}ɛ}{n}\mspace{14mu}{bits}\text{/}{token}}} & (19)\end{matrix}$

In equation (19), |C| is the cardinality of the set C and n is thenumber of tokens in the frame. Equation (19) indicates that eachadditional context can add a normalized cost of at least 11ε/n bits pertoken. The right-hand-side expression of equation (19) grows linearlywith the number of contexts (i.e., the size of the set C). Thus,reducing the number of contexts (i.e., a smaller set C) can reduce theoverhead. However, it remains the case that for every context, 11probabilities are encoded: the 11 corresponds to the number of tokens(12) minus one (1). Using selective mixing for entropy coding, asdescribed below, the number of probabilities to be encoded for somecontexts can be reduced. For example, given the coefficient token tree700, instead of encoding 11 probabilities, less than 11 are codedthereby reducing the overhead bits associated with coding distributionsin frame headers.

As mentioned above, reducing the number of contexts (i.e., a smaller setC) can reduce the overhead. However, analysis of the first term ofequation (18), namely the entropy H(Q_(c)), indicates that reducing thenumber of contexts is not desirable.

The entropy H(Q_(c)) is a concave function. This in turns means that,and as given by inequality (20), taking the linear combination of twodistributions M and M′, the entropy of the linear combination of the twodistributions M and M′ (i.e., the left-hand-side of the inequality) isgreater than or equal to the linear sum of the entropies of theindividual distributions (i.e., the right-hand-side of the inequality).The inequality (20) becomes a strict inequality when the twodistributions M and M′ are different.H(λM+(1−λ)M′)≥λH(M)+(1−λ)H(M′)  (20)

Given the inequality (20), it can be concluded that, in order tominimize the first term of the equation (18), it is desirable toincrease the number of distinct distributions, which in turn meansincreasing the number of distinct contexts. This is so because byincreasing the number of contexts, the left-hand-side of the inequality(20) can be decomposed into the right-hand-side. The decomposition canimprove the overall compression performance.

To summarize, in order to improve compression performance, analysis ofthe second term of equation (18) and equation (19) leads to theconclusion that it is preferable to reduce the number of contexts; onthe other hand, analysis of the first term of equation (18) andinequality (20) leads to the conclusion that it is desirable to increasethe number of contexts in order to improve compression performance.Using selective mixing for entropy coding, as described below, thenumber of contexts can be increased and the overhead associated withevery additional context can be reduced or limited. For example, andreferring to the coefficient token tree 700 of FIG. 7, adding a contextdoes not result in adding 11 probability values.

The following observations can be made:

-   -   1) In coding a transform coefficient, when a context c is        determined, the coding distribution P_(c) associated with the        context c is used to code token for the transform coefficient,    -   2) The multiplicative factor of 11 in equation (19) is equal to        the size of the alphabet of coefficient tokens minus one (1),    -   3) If a context c is to be split into two distinct contexts c₀        and c₁, then the number of times that the context c₀ appears and        the number of times that the context c₁ appears is equal to the        number of times that the context c appears (i.e., n_(c)=n_(c) ₀        +n_(c) ₁ ).    -   4) If a context c is to be split into two distinct contexts c₀        and c₁, then the corresponding actual distributions Q_(c) ₀ and        Q_(c) ₁ should be sufficiently different, and    -   5) For a given context c, the number of times n_(c,i) that the        token i appears in the frame under the context c, for all i,        should be sufficiently large to ensure sufficient accuracy in        probability estimation.

Given two distributions Q_(c) ₀ and Q_(c) ₁ that differ only at someindices but are similar or are the same at other indices,implementations according to this disclosure can split a context c into,for example, two contexts c₀ and c₁, only for those token indices atwhich Q_(c) ₀ and Q_(c) ₁ are different. In other words, each newcontext introduced can cost less than 11ε/n_(c) bits/token. For example,if a context is good for 4 tokens, then the cost is 3ε/n_(c).

FIG. 13 is a flowchart diagram of a process 1300 for coding transformcoefficients using an alphabet of transform coefficient tokens accordingto an implementation of this disclosure. The process 1300 codes (i.e.,encodes or decodes) a current transform coefficient using two or moreprobability distributions. The process 1300 can code a token indicativeof the current transform coefficient. The token can be determined orselected using a coefficient tree such as the coefficient token tree 700of FIG. 7. As such, the alphabet includes the leaf nodes (i.e., thetokens) of the coefficient tree.

The process 1300 can be used by, or in conjunction with, a process thatcodes the coefficients of a transform block according to a scan order.The current transform coefficient can be at a scan position i in thescan order and can be at a coefficient location (r_(i), c_(i)) in thetransform block.

The process 1300 selects a first probability distribution correspondingto a first context, selects a second probability distributioncorresponding to a second context, and can mix, for some transformcoefficient tokens, the first and second probability distributions togenerate a mixing probability for coding a transform coefficient tokenof the some transform coefficient tokens.

The process 1300 can be implemented in an encoder such as the encoder400 of FIG. 4. The process 1300 can be implemented, for example, as asoftware program that can be executed by computing devices such astransmitting station 102. The software program can includemachine-readable instructions that can be stored in a memory such as thememory 204 or the secondary storage 214, and that can be executed by aprocessor, such as CPU 202, to cause the computing device to perform theprocess 1300. In at least some implementations, the process 1300 can beperformed in whole or in part by the entropy encoding stage 408 of theencoder 400 of FIG. 4.

The process 1300 can be implemented in a decoder such as the decoder 500of FIG. 5. The process 1300 can be implemented, for example, as asoftware program that can be executed by computing devices such asreceiving station 106. The software program can include machine-readableinstructions that can be stored in a memory such as the memory 204 orthe secondary storage 214, and that can be executed by a processor, suchas CPU 202, to cause the computing device to perform the process 1300.In at least some implementations, the process 1300 can be performed inwhole or in part by the entropy decoding stage 502 of the decoder 500 ofFIG. 5.

When the process 1300 is implemented by an encoder, “coding” meansencoding in an encoded bitstream, such as the compressed bitstream 420of FIG. 4. When implemented by a decoder, “coding” means decoding froman encoded bitstream, such as the compressed bitstream 420 of FIG. 5.

At 1302, the process 1300 selects a first probability distribution. Asused in this disclosure, “select” means to obtain, identify, construct,determine, specify or other select in any manner whatsoever.

In an example, the process 1300 can first derive a first context andselect the first probability distribution that corresponds to the firstcontext. For example, the context can be derived using one or more ofthe transform block size, the transform block shape (e.g., square orrectangular), the color component or plane type (i.e., luminance orchrominance), the scan position i of the current transform coefficient,and previously coded tokens. For example, in the case of the scan order602 of FIG. 6, the previously coded coefficients can be the leftneighboring coefficient and the top neighboring coefficient of thecurrent transform coefficient. Other information can be used to derivethe first context.

In an example, the first probability distribution can be defined overall the tokens of the alphabet. That is, the probability distributionincludes a probability value for each of the tokens of the alphabet.Using the tokens of the coefficient token tree 700, let the alphabet setE denote the alphabet of coefficient tokens. As such the alphabet set Eis given by E={EOB_TOKEN, ZERO_TOKEN, ONE_TOKEN, TWO_TOKEN, THREE_TOKEN,FOUR_TOKEN, DCT_VAL_CAT1, DCT_VAL_CAT2, DCT_VAL_CAT3, DCT_VAL_CAT4,DCT_VAL_CAT5, DCT_VAL_CAT6}. The first probability distribution caninclude a probability value for each of the tokens of the alphabet setE. In another example, the probability distribution can includeprobability values for some of the tokens of the alphabet. In anexample, the first probability distribution can be a coding distributionas described above with respect to the coding distribution P_(c).

At 1304, the process 1300 selects a second probability distribution. Inan example, the process 1300 can derive a second context and select thesecond probability distribution that corresponds to the second context.

The second probability distribution can be defined over a partition ofthe tokens. Using the tokens of the coefficient token tree 700 of FIG. 7as an illustrative example, the partition can correspond to anon-trivial partition F_(E) of the alphabet set E. For example, thenon-trivial partition F_(E) can be F_(E)={{EOB_TOKEN}, {ZERO_TOKEN},{ONE_TOKEN, TWO_TOKEN, THREE_TOKEN, FOUR_TOKEN, DCT_VAL_CAT1,DCT_VAL_CAT2, DCT_VAL_CAT3, DCT_VAL_CAT4, DCT_VAL_CAT5, DCT_VAL_CAT6}}.That is, the non-trivial partition F_(E) partitions the alphabet set Einto three non-overlapping subsets: {EOB_TOKEN}, {ZERO_TOKEN}, and theset that includes all other tokens. As such, the second probabilitydistribution includes probability values for the elements of thepartition.

In the example of the non-trivial partition F_(E), the secondprobability distribution can include three (3) probability values. Asthe non-trivial partition F_(E) includes only three (3) elements, thesecond context adds 2ε/n bits/token of overhead, which is significantlysmaller than that of about 11ε/n bits/token added by a new contextdetermining a probability distribution over the alphabet E (i.e., acontext such as the first context).

As the second context targets a subset of the tokens of the alphabet E,the amount of overhead associated with an added second context islimited. An added second context results in much less overhead than anadded first context.

In an example, the second context can be a context that targets thetokens of the alphabet set E that are used more frequently than othernodes. Targeting the more frequently used tokens can improve the coding(e.g., compression) performance for those tokens. For example, andreferring again to the coefficient token tree 700 of FIG. 7, of theinternal nodes (e.g., the nodes A-K), the root node 701 and the node 703are the most frequently used nodes when coding transform coefficients.In traversing the tree to encode transform coefficients, the furtherdown the tree an internal node is, the less frequently the node istraversed. That is, the further down the tree an internal node is, thesmaller the number of times that the internal node is used in coding atransform coefficient becomes. As such, adding contexts (which is, asdescribed above, desirable but for the overhead), can be limited toadding contexts for the most frequently used tokens.

In an example, the second context can be an actual distribution asdescribed with respect to the actual distribution Q_(c). In an example,the second context can be derived by leveraging the well-known fact thatthe probability of a token for the current coefficient (i.e., thecoefficient at coefficient location (r_(i), c_(i)) and corresponding tothe scan position i) being zero (i.e., t_(i)=ZERO_TOKEN) is stronglycorrelated with the number of zeros in an immediate 2D (two-dimensional)neighborhood of the coefficient location (r_(i), c_(i)). In an example,the second context can be derived using a neighborhood template that isanchored at the coefficient location (r_(i), c_(i)). A neighborhoodtemplate can indicate, include, specify, or the like, the coefficientlocations that constitute the neighborhood. The neighborhood templatecan be based on a scan order.

FIG. 14 is a diagram of neighborhood templates 1400 and 1450 forderiving a context according to implementations of this disclosure. Aneighborhood template includes the locations (i.e., neighboringlocations) of coefficients that are coded before the currentcoefficient. As such, values for those coefficients are available forcoding the current coefficient. The neighborhood template can includeany number of locations. The neighborhood template can have any shape.For example, the neighborhood template need not include contiguous oradjacent locations.

The neighborhood template 1400 illustrates a neighborhood template thatcan be used with a forward scan order. A forward scan order is a scanorder that proceeds from the top-left corner of the transform block tothe bottom-right corner, such as the scan order 602 of FIG. 6. Theneighborhood template 1400 illustrates a current coefficient 1402 and aneighborhood template 1404 that includes the locations of the fiveshaded coefficients marked a-e. A neighborhood template can include moreor less coefficient locations. In an example, the values a-e canindicate whether the respective coefficients are zero or non-zero. Assuch, the values a-e can be binary values.

The neighborhood template 1450 illustrates a neighborhood template thatcan be used with a backward scan order. A backward scan order is a scanorder that proceeds, for example, from the bottom-right corner of thetransform block to the top-left corner. The neighborhood template 1450illustrates a current coefficient 1452 and a neighborhood template 1454that includes the locations of the nine shaded coefficients labeled a-i.However, as indicated above, a neighborhood template can include more orless coefficient locations.

In an example, the second context can be derived based on the number ofzero coefficients in the neighborhood template. For example, and usingthe neighborhood template 1400, a context value can be selected from theset of context values {0, 1, 2, 3} based on the formula (a+b+c+d+e+1)>>1where each of the values a-e is a zero (0) or a one (1) value and the“>>1” is a bit-shifts the sum (a+b+c+d+e+1) by one bit. In an example, avalue of zero (0) can indicate that the coefficient at that location isa zero coefficient and a value of one (1) can indicate that thecoefficient is non-zero. For example, if all of the neighborhoodtemplate coefficients are zero, then the context numbered 0 can beselected; if exactly one or exactly two of the neighborhood templatecoefficients are non-zero, then the context numbered 1 is selected; andso on. Other values and semantics are possible.

Referring again to FIG. 13, at 1306, in response to determining that thesecond probability distribution includes a probability for a transformcoefficient token, the process 1300 proceeds to 1308. In animplementation, the process 1300 can include, if the second probabilitydistribution does not include a probability for the transformcoefficient token, then the process 1330 proceeds to 1312.

In an example, the second probability distribution includes aprobability for a transform coefficient token when the transformcoefficient token is included in a singleton element of the partition.For example, the non-trivial partition F_(E) described above isdetermined to include a probability for the token EOB_TOKEN since theEOB_TOKEN is included is the singleton element {EOB_TOKEN} of thenon-trivial partition F_(E). As a reminder, a singleton element is asubset of the alphabet set E that includes only one element. As such,the second probability distribution is not determined to include aprobability for, e.g., the FOUR_TOKEN because the FOUR_TOKEN is notincluded in a singleton element of the non-trivial partition F_(E).

As indicated above, the first context can be used to obtain the firstprobability distribution that can be a probability distribution over thealphabet set E and the second context can be used to obtain a secondprobability distribution that is defined over the non-trivial partitionF_(E). Referring to the coefficient token tree 700, in an example, thefirst context can be used to determine a binary probability distributionfor coding (i.e., encoding or decoding) binary decisions at everyinternal node, and the second context can be used to determine binarydistributions at only two internal nodes: the root node 701 and itsright child node (i.e., the node 703).

At 1308, the process 1300 mixes the first probability distribution andthe second probability distribution to generate a mixed probability. At1310, the process 1330 entropy codes the transform coefficient tokenusing the mixed probability.

Given the non-trivial partition F_(E), the coding distributions for thetwo internal nodes (i.e., the root node 701 and the node 703) may beobtained by mixing the distribution obtained from the first context(i.e., the first probability distribution) and the distribution obtainedfrom the second context (i.e., the second probability distribution). Assuch, a probability distribution for coding a token need not be selecteda priori; rather the mixing, as described above, can result in the bestcombination.

The mixing can be as described above with respect to FIGS. 9-12. In anexample, the process 1300 can generate the mixed probability bydetermining, using the first probability distribution, a firstconditional probability for decoding the transform coefficient token,determining, using the second probability distribution, a secondconditional probability for encoding the transform coefficient token,and determining, using the first conditional probability and the secondconditional probability, the mixed probability.

Using a token tree, such as the coefficient token tree 700 of FIG. 7, asan example, the first conditional probability distribution can be theconditional probability distribution at an internal node. Theconditional probability distribution at an internal node is theprobability distribution of selecting a child node (i.e., the left childor the right child) of the internal node given a first contextdetermined from coded history (i.e., previously coded coefficients) andside information. Examples of side information include plane type (e.g.,luminance, chrominance, etc.), transform size (i.e., transform blocksize), and transform type (e.g., DCT, ADST, etc.). Other sideinformation can be available. The second conditional probabilitydistribution is the conditional probability distribution at the sameinternal node given a second context determined from coded history andside information. The first context and the second context can bederived using different information in the coded history and differentside information.

Using the non-trivial partition F_(E) as an example, the firstconditional probability distribution can be the conditional probabilitydistribution over the non-trivial partition F_(E) given a first contextdetermined from coded history and side information; and the secondconditional probability distribution can be the conditional probabilitydistribution over F_(E) given a second context determined from codedhistory and other or the same side information.

If a first context determines a probability distribution P, over thealphabet set E, a probability distribution Q in F_(E) can be determinedsuch that, for any element e in F_(E), the probability of the element e,Q(e), can be given by the sum of the probabilities of all the tokens inthe element e (e a set of tokens): Q(e)=Σ_(t∈e)P (t). Since F_(E) is anon-trivial partition of the alphabet set E, selective mixing ofprobability values is essentially carried out or performed for elementse in the alphabet set E.

At 1312, on condition that the second probability distribution notincluding a probability for the transform coefficient token, the process1300 uses the first probability distribution to entropy code the currenttransform coefficient token. That is, for the remaining internal nodes(i.e., the nodes that are not included in singletons of the partitionE), the coding distributions may be obtained from the first context.That is, the first probability distribution can be used to entropy codethe remaining coefficients.

To summarize, selective mixing of the first probability distribution andthe second probability distribution can be used for a subset of tokensused for coding transform coefficients. In an example, the tokenscorrespond to the internal nodes of a coefficient token tree. Forexample, selective mixing can be used only for internal nodes (i.e.,often-used nodes) that are used more often than other internal nodes(i.e., not often-used nodes). In an example, the internal nodes that areused more often can be given as a listing of the tokens corresponding tothose internal nodes. In another example, internal nodes that are usedmore often can be the singletons of the non-trivial partition F_(E). Ifmixing is not used for an internal node, then the first probabilitydistribution can be used as the coding distribution. As such, mixing isselectively applied to some nodes: the first and the seconddistributions are mixed for some internal nodes (e.g., an often-usednode) and the first distribution is used for coding the other internalnodes (e.g., a not often-used node).

In the above example, the alphabet set E was partitioned into thenon-trivial partition F_(E). As such, the token of the alphabet set Eare considered unrelated and distinct. That is, even though the tokensof the coefficient token tree 700 were used to illustrate the process1300, the partition treats the tokens in unrelated and distinct tokens.That is, the partition E does not leverage the tree structure and can beused with any alphabet set.

In another example, a tree structure of tokens, if available, can beused to select the second probability distribution. The tree structure(such as the coefficient token tree 700) can relate, by virtue of itshierarchy, one token to another. As such, the second probabilitydistribution can be a probability distribution that is defined at someinternal nodes of the coefficient token tree.

In an example, the internal nodes that are used more often can be nodesthat are closer to the root node than other internal nodes. For example,in the coefficient token tree 700, the node labeled C is closer to theroot node 701 than the node labeled K because traversal from the rootnode requires less hops to reach the node labeled C than to reach thenode labeled K.

In an example, selective mixing can be used for internal nodes that arewithin (or no more than) a predetermined number of hops from the rootnode. For example, if the predetermined number of hops is two (2), thenselective mixing can be used for the internal nodes labeled A (i.e., theroot node 701), the node 703, and the node labeled C. As such, whetheran internal is “often used” can be determined based a proximity of theinternal node to the root node. An internal node, corresponding to atoken, is “used” when, for example, in the process of coding anothertoken, a decision related to the token is also coded.

The probability of coding a token of the some internal nodes (i.e., asequence x^(n) generated by traversing the tree) can be determined asdescribed with respect to FIG. 9 by mixing the first probabilitydistribution and the second probability distribution. That is, for theinternal nodes that have both distributions from the first probabilitydistribution and the second probability distribution, a mixedprobability can be obtained for entropy coding the current coefficient.For all other internal nodes, the first probability distribution is usedto entropy code the current coefficient.

If k is the number of internal nodes that are affected by the secondcontext, then the second context adds, approximately, kε/n bits/token,whereas the first context adds, for the tokens of the coefficient tokentree 700, 11ε/n bits/token.

By using selective mixing, the set of available contexts C in codingsystem can be divided into a first set C₀ and a second set C₁. Using thealphabet E of coefficient tokens as an example, the first set C₀ can bederived from contextual information that affects the overalldistribution over the alphabet E, and contexts in the set C₁ can bederived from contextual information that affects only a portion of thedistribution over the alphabet E. Different contexts in the set C₁ cantarget different partitions of the alphabet E. Where a coefficient treeis available, different contexts in the set C₁ can target differentinternal nodes of the tree. For example, some contexts in the set C₁ cantarget the root node in the coefficient token tree. For example, somecontexts in the set C₁ can target the internal node splitting ZERO_TOKENfrom other tokens. For example, some contexts in the set C₁ can targetthe internal node splitting ONE_TOKEN from other tokens. Accordingly,instead of maintaining hundreds or thousands of contexts for all tokensof an alphabet in a coding system, a smaller subset can be maintainedfor all the tokens and another set of contexts can target tokens thatmay be deemed important, significant, or more frequently used.

The aspects of encoding and decoding described above illustrate someencoding and decoding techniques. However, it is to be understood thatencoding and decoding, as those terms are used in the claims, could meancompression, decompression, transformation, or any other processing orchange of data.

The words “example” or “implementation” are used herein to mean servingas an example, instance, or illustration. Any aspect or design describedherein as “example” or “implementation” is not necessarily to beconstrued as preferred or advantageous over other aspects or designs.Rather, use of the words “example” or “implementation” is intended topresent concepts in a concrete fashion. As used in this application, theterm “or” is intended to mean an inclusive “or” rather than an exclusive“or.” That is, unless specified otherwise, or clear from context, “Xincludes A or B” is intended to mean any of the natural inclusivepermutations. That is, if X includes A; X includes B; or X includes bothA and B, then “X includes A or B” is satisfied under any of theforegoing instances. In addition, the articles “a” and “an” as used inthis application and the appended claims should generally be construedto mean “one or more” unless specified otherwise or clear from contextto be directed to a singular form. Moreover, use of the term “animplementation” or “one implementation” throughout is not intended tomean the same embodiment or implementation unless described as such.

Implementations of transmitting station 102 and/or receiving station 106(and the algorithms, methods, instructions, etc., stored thereon and/orexecuted thereby, including by encoder 400 and decoder 500) can berealized in hardware, software, or any combination thereof. The hardwarecan include, for example, computers, intellectual property (IP) cores,application-specific integrated circuits (ASICs), programmable logicarrays, optical processors, programmable logic controllers, microcode,microcontrollers, servers, microprocessors, digital signal processors orany other suitable circuit. In the claims, the term “processor” shouldbe understood as encompassing any of the foregoing hardware, eithersingly or in combination. The terms “signal” and “data” are usedinterchangeably. Further, portions of transmitting station 102 andreceiving station 106 do not necessarily have to be implemented in thesame manner.

Further, in one aspect, for example, transmitting station 102 orreceiving station 106 can be implemented using a general purposecomputer or general purpose processor with a computer program that, whenexecuted, carries out any of the respective methods, algorithms and/orinstructions described herein. In addition, or alternatively, forexample, a special purpose computer/processor can be utilized which cancontain other hardware for carrying out any of the methods, algorithms,or instructions described herein.

Transmitting station 102 and receiving station 106 can, for example, beimplemented on computers in a video conferencing system. Alternatively,transmitting station 102 can be implemented on a server and receivingstation 106 can be implemented on a device separate from the server,such as a hand-held communications device. In this instance,transmitting station 102 can encode content using an encoder 400 into anencoded video signal and transmit the encoded video signal to thecommunications device. In turn, the communications device can thendecode the encoded video signal using a decoder 500. Alternatively, thecommunications device can decode content stored locally on thecommunications device, for example, content that was not transmitted bytransmitting station 102. Other transmitting station 102 and receivingstation 106 implementation schemes are available. For example, receivingstation 106 can be a generally stationary personal computer rather thana portable communications device and/or a device including an encoder400 may also include a decoder 500.

Further, all or a portion of implementations of the present disclosurecan take the form of a computer program product accessible from, forexample, a tangible computer-usable or computer-readable medium. Acomputer-usable or computer-readable medium can be any device that can,for example, tangibly contain, store, communicate, or transport theprogram for use by or in connection with any processor. The medium canbe, for example, an electronic, magnetic, optical, electromagnetic, or asemiconductor device. Other suitable mediums are also available.

The above-described embodiments, implementations and aspects have beendescribed in order to allow easy understanding of the present disclosureand do not limit the present disclosure. On the contrary, the disclosureis intended to cover various modifications and equivalent arrangementsincluded within the scope of the appended claims, which scope is to beaccorded the broadest interpretation so as to encompass all suchmodifications and equivalent structure as is permitted under the law.

What is claimed is:
 1. An apparatus for decoding transform coefficientsusing an alphabet of transform coefficient tokens, the apparatuscomprising: a memory; and a processor configured to execute instructionsstored in the memory to: select, for entropy decoding a transformcoefficient token, a first probability distribution corresponding to afirst context, the first probability distribution being defined for alltokens of the alphabet, wherein the first context does not include thetransform coefficient token; select, for entropy decoding the transformcoefficient token, a second probability distribution corresponding to asecond context, the second probability distribution being defined over anon-trivial partition of the tokens, wherein the non-trivial partitionof the tokens partitions the alphabet into more than one non-overlappingsubsets of the tokens, and wherein the second context does not includethe transform coefficient token; and in response to determining that thesecond probability distribution includes a probability for the transformcoefficient token by determining that the transform coefficient token isincluded in a singleton element of the non-trivial partition, executinginstructions to: mix the first probability distribution and the secondprobability distribution to generate a mixed probability; and entropydecode, from an encoded bitstream, the transform coefficient token usingthe mixed probability.
 2. The apparatus of claim 1, wherein theinstructions further comprise instructions to: on condition that thesecond probability distribution not including a probability for thetransform coefficient token, use the first probability distribution toentropy decode the transform coefficient token from the encodedbitstream.
 3. The apparatus of claim 1, wherein the first probabilitydistribution is a coding distribution.
 4. The apparatus of claim 1,wherein the second probability distribution is an actual distribution.5. The apparatus of claim 1, wherein to mix the first probabilitydistribution and second probability distribution to entropy decode thetransform coefficient token from the encoded bitstream comprises to:determine, using the first probability distribution, a first conditionalprobability for decoding the transform coefficient token, the firstconditional probability being a conditional probability of the transformcoefficient token given other transform coefficient tokens of thealphabet; determine, using the second probability distribution, a secondconditional probability for encoding the transform coefficient token,the second conditional probability being a conditional probability ofthe singleton element of the non-trivial partition given other elementsof the non-trivial partition; and determine, using the first conditionalprobability and the second conditional probability, the mixedprobability for decoding the transform coefficient token.
 6. Theapparatus of claim 1, wherein a transform coefficient corresponding tothe transform coefficient token is at a location of a transform block,and wherein the second context is determined using a number of zerocoefficients at locations neighboring the location.
 7. The apparatus ofclaim 6, wherein the locations neighboring the location are based on ascan order.
 8. A method for coding transform coefficients using analphabet of tokens, comprising: selecting, for entropy coding a tokencorresponding to a transform coefficient, a first probabilitydistribution corresponding to a first context, the first probabilitydistribution being defined for some tokens of the alphabet; selecting,for entropy coding the token, a second probability distributioncorresponding to a second context, the second probability distributionbeing defined over a non-trivial partition of the tokens, wherein thenon-trivial partition of the tokens partitions the alphabet into morethan one non-overlapping subsets of the tokens; and in response todetermining that the first probability distribution includes aprobability for the token and the second probability distributionincludes a second probability for the token, mixing the firstprobability distribution and the second probability distribution togenerate a mixed probability, and coding the token using the mixedprobability.
 9. The method of claim 8, further comprising: on conditionthat the second probability distribution not including a probability forthe token, using the first probability distribution to entropy code thetoken.
 10. The method of claim 8, wherein the first probabilitydistribution is a coding distribution.
 11. The method of claim 8,wherein the second probability distribution is an actual distribution.12. The method of claim 8, wherein determining that the secondprobability distribution includes a probability for the token comprises:determining that the token is included in a singleton element of thenon-trivial partition.
 13. The method of claim 12, wherein mixing thefirst probability distribution and second probability distribution toentropy code the token comprises: determining, using the firstprobability distribution, a first conditional probability for decodingthe token, the first conditional probability being a conditionalprobability of the token given other tokens of the alphabet;determining, using the second probability distribution, a secondconditional probability for encoding the token, the second conditionalprobability being a conditional probability of the singleton element ofthe non-trivial partition given other elements of the non-trivialpartition; and determining, using the first conditional probability andthe second conditional probability, the mixed probability for decodingthe token.
 14. The method of claim 8, wherein the transform coefficientis at a location, and wherein the second context is determined using anumber of zero coefficients at neighboring locations.
 15. The method ofclaim 14, wherein the neighboring locations are based on a scan order.16. An apparatus for decoding transform coefficients using an alphabetof tokens organized in a coefficient token tree, the apparatuscomprising: a memory; and a processor configured to execute instructionsstored in the memory to: select a first probability distributioncorresponding to a first context, the first probability distributionbeing defined for internal nodes of the coefficient token tree; select asecond probability distribution corresponding to a second context, thesecond probability distribution being defined for some, but not all,internal nodes of the coefficient token tree; decode a first decisionrelated to a first internal node of the coefficient token tree using amixed probability, the mixed probability generated by mixing the firstprobability distribution and the second probability distribution; anddecode a second decision related to a second internal node of thecoefficient token tree using the first probability distribution, whereinthe first internal node is used more often than the second internalnode.
 17. The apparatus of claim 16, wherein the first internal node isindicative of an end-of-block.